An automatic building façade deterioration detection system using infrared-visible image fusion and deep learning

Diverse building fa & ccedil;ade deteriorations occurring both externally and internally within the materials have emerged as substantial challenges to the structural durability and the occupant safety. Nevertheless, prevailing evaluation and detection endeavors for these deteriorations have predominantly hinged on the scrutiny of surface-level visual data, overlooking the integrative potential offered by the infrared imagery that unveils deteriorations transpiring at certain depths, including those associated with moisture and plaster detachment. Therefore, this study proposes a novel hybrid method for automatic building fa & ccedil;ade deterioration detection by seamlessly integrating the cross-referenced infrared and visible images using deep learning. A dataset comprising 1228 pairs of infrared and visible images, representing four key deteriorations-crack, spalling, moisture-related damage, and plaster detachment-is collected for training and validation. An infrared-visible image fusion (IVIF) module based on the generative adversarial network (GAN) is subsequently trained to concurrently preserve deterioration characteristics evident in either of the image modalities. Four instance segmentation models are trained and compared afterwards. The outcomes substantiate the accomplished IVIF method, validated through both high-performing qualitative and quantitative assessments. The noteworthy high mean average precision (mAP) result of 86.5 % obtained through the subsequent instance segmentation module affirm a thorough utilization of the complementary information, thereby enhancing decision-making processes crucial for the maintenance of building fa & ccedil;ades throughout their service life.