Combination of pixel-wise and region-based deep learning for pavement inspection and segmentation

This paper presents a new model combining pixel-wise and region-based deep learning to provide a pavement inspection technology for jointly obtaining the distress classes, locations, and geometric information. This proposed model, called the segmentation RCNN, added an extract branch in a faster region convolutional neural network (Faster RCNN) for assigning the pixels in each region of interest (ROI) from an image into one of the pavement distresses or background, in parallel with the existing branches for ROI classification and bounding-box regression in the Faster RCNN. The effectiveness of the proposed model was tested by a pavement-image database collected from 16 asphalt pavements. The results indicated that the proposed model detected and segmented the pavement distresses (cracks, potholes, and patches) with mean intersection over unions of 87.6% and 70.3%, respectively. The proposed model was stable under various real-world conditions. The model reduced the computation costs, which provided a novel direction to achieve real-time pavement inspection.