Automatic pavement crack detection using multimodal features fusion deep neural network

The existing pavement crack detection algorithms are all dealing with detection and segmentation separately, ignoring the feature correlation between the bounding box coordinates and mask information, and there are still problems such as low crack detection accuracy, incomplete detection, and segmentation fracture in practical application. In view of the above situation, this paper took the detection bounding box coordinates and mask information as multimodal features of the same pavement crack area, and proposed a One-Stage MFFNet (Multimodal Feature Fusion Network), which improved the pavement crack detection accuracy and segmentation integrity obviously. And experimental results of different models were compared on self-collected datasets and two public datasets (CFD and CRACK500) respectively. Compared with Mask R-CNN, the average detection accuracy and average segmentation accuracy were improved by 2.6% and 4.7%, respectively. Compared with the optimised model of RDSNet, the detection accuracy and processing speed were improved by 1.8% and 2FPS, respectively. In addition, MFFNet had significantly improved the integrity of pavement crack segmentation results. The results showed that the proposed MFFNet model achieved the best detection and segmentation accuracy, and was an effective and high-precision pavement crack detection model.