Dual-branch collaborative transformer for effective

被引:0
|
作者
Qi, Xuanyu [1 ]
Song, Tianyu [1 ]
Dong, Haobo [1 ]
Jin, Jiyu [1 ]
Jin, Guiyue [1 ]
Li, Pengpeng [1 ]
机构
[1] Dalian Polytech Univ, Sch Informat Sci & Engn, Dalian 116034, Peoples R China
关键词
Single image deraining; Transformer; Multi-dimensional features; Feature fusion; RAIN STREAKS REMOVAL; IMAGE;
D O I
10.1016/j.jvcir.2024.104136
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Recently, Transformer -based architecture has been introduced into single image deraining tasks due to its advantage in modeling non -local information. However, existing approaches typically utilize self -attention along single spatial or channel dimension. They neglect feature fusion in different dimensions to explore contextual information fully, which limits the effective receptive field of the network and makes it challenging to learn image degradation relationships. To fully explore potential correlations between different dimensions of degraded images, we develop a D ual -branch C ollaborative Trans former , called DCformer. To be specific, we employ parallel multi -head self -attention (PMSA) as the core block to extract long-range contextual relationships across spatial and channel dimensions. Additionally, a local perception block (LPB) is introduced to provide the ability of local information acquisition in the network, which is complementary to the global modeling ability of the parallel hybrid self -attention mechanism. Finally, we design a feature interaction block (FIB) to further enhance the interaction of features at different resolutions. Extensive experiments on benchmark datasets demonstrate the effectiveness of our proposed method.
引用
收藏
页数:9
相关论文
共 50 条
  • [1] Dual-Branch Collaborative Transformer for Virtual Try-On
    Fenocchi, Emanuele
    Morelli, Davide
    Cornia, Marcella
    Baraldi, Lorenzo
    Cesari, Fabio
    Cucchiara, Rita
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2022, 2022, : 2246 - 2250
  • [2] DDT: Dual-branch Deformable Transformer for Image Denoising
    Liu, Kangliang
    Du, Xiangcheng
    Liu, Sijie
    Zheng, Yingbin
    Wu, Xingjiao
    Jin, Cheng
    [J]. 2023 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME, 2023, : 2765 - 2770
  • [3] Dual-branch network based on transformer for texture recognition
    Liu, Yangqi
    Dong, Hao
    Wang, Guodong
    Chen, Chenglizhao
    [J]. DIGITAL SIGNAL PROCESSING, 2024, 153
  • [4] Dual-branch collaborative Siamese network for visual tracking
    Zhou, Wenjun
    Liu, Yao
    Wang, Nan
    Wang, Yifan
    Peng, Bo
    [J]. DIGITAL SIGNAL PROCESSING, 2024, 155
  • [5] Dual-Branch Collaborative Learning for Visual Question Answering
    Tian, Weidong
    Zhao, Junxiang
    Xu, Wenzheng
    Zhao, Zhongqiu
    [J]. ADVANCED INTELLIGENT COMPUTING TECHNOLOGY AND APPLICATIONS, PT III, ICIC 2024, 2024, 14864 : 96 - 107
  • [6] A Dual-Branch Multiscale Transformer Network for Hyperspectral Image Classification
    Shi, Cuiping
    Yue, Shuheng
    Wang, Liguo
    [J]. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 20
  • [7] DBT: multimodal emotion recognition based on dual-branch transformer
    Yufan Yi
    Yan Tian
    Cong He
    Yajing Fan
    Xinli Hu
    Yiping Xu
    [J]. The Journal of Supercomputing, 2023, 79 : 8611 - 8633
  • [8] Dual-Branch Adaptive Convolutional Transformer for Hyperspectral Image Classification
    Wang, Chuanzhi
    Huang, Jun
    Lv, Mingyun
    Wu, Yongmei
    Qin, Ruiru
    [J]. REMOTE SENSING, 2024, 16 (09)
  • [9] DBT: multimodal emotion recognition based on dual-branch transformer
    Yi, Yufan
    Tian, Yan
    He, Cong
    Fan, Yajing
    Hu, Xinli
    Xu, Yiping
    [J]. JOURNAL OF SUPERCOMPUTING, 2023, 79 (08): : 8611 - 8633
  • [10] Dual-branch vision transformer for blind image quality assessment*
    Lee, Se-Ho
    Kim, Seung-Wook
    [J]. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2023, 94