Defect inspection of indoor components in buildings using deep learning object detection and augmented reality

Visual inspection is commonly adopted for building operation, maintenance, and safety. The durability and defects of components or materials in buildings can be quickly assessed through visual inspection. However, implementations of visual inspection are substantially time-consuming, labor-intensive, and error-prone because useful auxiliary tools that can instantly highlight defects or damage locations from images are not available. Therefore, an advanced building inspection framework is developed and implemented with augmented reality (AR) and real-time damage detection in this study. In this framework, engineers should walk around and film every corner of the building interior to generate the three-dimensional (3D) environment through ARKit. Meanwhile, a trained YOLOv5 model real-time detects defects during this process, even in a large-scale field, and the defect locations indicating the detected defects are then marked in this 3D environment. The defects areas can be measured with centimeter-level accuracy with the light detection and ranging (LiDAR) on devices. All required damage information, including defect positions and sizes, is collected at a time and can be rendered in the 2D and 3D views. Finally, this visual inspection can be efficiently conducted, and the previously generated environment can also be loaded to re-localize existing defect marks for future maintenance and change observation. Moreover, the proposed framework is also implemented and verified by an underground parking lot in a building to detect and quantify surface defects on concrete components. As seen in the results, the conventional building inspection is significantly improved with the aid of the proposed framework in terms of damage localization, damage quantification, and inspection efficiency.