RainFormer: a pyramid transformer for single image deraining

Rain impairs the performance of outdoor vision systems, such as automated driving systems and outdoor surveillance systems. Therefore, as an image preprocessing technique, image deraining has great potential for application. Defects of convolutional neural networks (small receptive field and non-adaptive to input content) limit the further improvement of deraining model performance. Recently, a novel neural network, transformer, has demonstrated impressive performance on natural language processing and vision tasks. However, using transformer for image deraining still has some issues: Although transformers have powerful long-range computing capabilities, it lacks the ability to model local features, which is critical for image deraining. In addition, transformer uses fixed-size patches to process images, which leads to pixels at the edges of the patches that cannot use the local features of neighboring pixels to restore rain-free images. In this paper, we propose a novel pyramid transformer for image deraining. To address the first issue, we design a residual-Dconv feed-forward network (RDFN), where depth-wise convolution improves the capability of modeling local features. To address the second issue, we introduce multi-resolution features into the transformer, which allows the transformer to obtain patches with different scales, thus enabling the boundary pixels to utilize local features. Furthermore, we propose a novel multi-scale fusion bridge (MSFB) to effectively integrate the extracted multi-scale features and capture the correlation between different scales. Extensive experiments on synthetic and real-world images demonstrate that the proposed deraining model achieves superior performance, especially the PSNR value achieves 47.55 dB on the SPA-Data dataset. We also further validate the effectiveness of the proposed model on subsequent high-level computer vision tasks.