Few-shot object detection and attribute recognition from construction site images for improved field compliance

Computer vision techniques can be used to detect site objects for identifying noncompliances that could lead to safety incidents. However, existing methods are limited in covering diverse hazard scenarios, detecting site objects with imbalanced distributions, and recognizing their intricate attributes to describe their conditions and functionality. To address these gaps, this paper proposes a deep learning-based method for identifying multiple fall-related objects and their associated attributes. The proposed method consists of three submethods: (1) a method for developing relevant datasets by retrieving images from open resources; (2) a method for few-shot object detection, which deals with imbalanced distributions; and (3) a method for attribute recognition to add semantic descriptions to the detected objects. The proposed method achieved an average precision and recall of 88.2% and 79.5% for few-shot object detection and 94.8% and 95.7% for attribute recognition, respectively, which indicates good performance.