Multiscale Nonlocal Neural Network for Single Image Deraining

Rainy day is a common severe weather, in which rain streaks seriously affect the accuracy of algorithms, such as object classification, detection, and segmentation. In a rain image, multiscale rain streaks have similar shape features, which make it possible to exploit such complementary information for the collaborative representation of rain streaks. In this study, we construct a multiscale feature pyramid structure to exploit the similarity features between different rain streaks and design the initial, convolutional long short-term memory network (Conv-LSTM), fusion, and reconstruction modules. In addition, we introduce a lightweight nonlocal mechanism in the fusion module to guide the fine fusion and removal of rain streak features. Extensive experiments were conducted on synthetic and real-world datasets. Compared with four recent deep learning-based single image deraining methods, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the proposed method significantly improved. Experimental results show that the proposed method can effectively remove rain streaks and avoid image blur, while maintaining the original image information.