Spatial relaxation transformer for image super-resolution

被引:0
|
作者
Li, Yinghua [1 ]
Zhang, Ying [1 ]
Zeng, Hao [3 ]
He, Jinglu [1 ]
Guo, Jie [2 ]
机构
[1] Xian Univ Posts & Telecommun, Xian Key Lab Image Proc Technol & Applicat Publ Se, Changan West St, Xian 710121, Shaanxi, Peoples R China
[2] Xidian Univ, State Key Lab Integrated Serv Networks, 2 Southern Tai Bai Rd, Xian 710071, Shaanxi, Peoples R China
[3] Chinese Acad Sci, Inst Software, Beijing, Peoples R China
关键词
Super-resolution; Vision transformer; Feature aggregation; Image enhancement; Swin transformer;
D O I
10.1016/j.jksuci.2024.102150
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Transformer-based approaches have demonstrated remarkable performance in image processing tasks due to their ability to model long-range dependencies. Current mainstream Transformer-based methods typically confine self-attention computation within windows to reduce computational burden. However, this constraint may lead to grid artifacts in the reconstructed images due to insufficient cross-window information exchange, particularly in image super-resolution tasks. To address this issue, we propose the Multi-Scale Texture Complementation Block based on Spatial Relaxation Transformer (MSRT), which leverages features at multiple scales and augments information exchange through cross windows attention computation. In addition, we introduce a loss function based on the prior of texture smoothness transformation, which utilizes the continuity of textures between patches to constrain the generation of more coherent texture information in the reconstructed images. Specifically, we employ learnable compressive sensing technology to extract shallow features from images, preserving image features while reducing feature dimensions and improving computational efficiency. Extensive experiments conducted on multiple benchmark datasets demonstrate that our method outperforms previous state-of-the-art approaches in both qualitative and quantitative evaluations.
引用
收藏
页数:10
相关论文
共 50 条
  • [1] Single Image Super-resolution Using Spatial Transformer Networks
    Wang, Qiang
    Fan, Huijie
    Cong, Yang
    Tang, Yandong
    [J]. 2017 IEEE 7TH ANNUAL INTERNATIONAL CONFERENCE ON CYBER TECHNOLOGY IN AUTOMATION, CONTROL, AND INTELLIGENT SYSTEMS (CYBER), 2017, : 564 - 567
  • [2] Spatial Transformer Generative Adversarial Network for Image Super-Resolution
    Rempakos, Pantelis
    Vrigkas, Michalis
    Plissiti, Marina E.
    Nikou, Christophoros
    [J]. IMAGE ANALYSIS AND PROCESSING, ICIAP 2023, PT I, 2023, 14233 : 399 - 411
  • [3] Transformer for Single Image Super-Resolution
    Lu, Zhisheng
    Li, Juncheng
    Liu, Hong
    Huang, Chaoyan
    Zhang, Linlin
    Zeng, Tieyong
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2022, 2022, : 456 - 465
  • [4] A spectral and spatial transformer for hyperspectral remote sensing image super-resolution
    Wang, Bingqian
    Chen, Jianhua
    Wang, Huajun
    Tang, Yipeng
    Chen, Jiongling
    Jiang, Ye
    [J]. INTERNATIONAL JOURNAL OF DIGITAL EARTH, 2024, 17 (01)
  • [5] A novel spatial and spectral transformer network for hyperspectral image super-resolution
    Wu, Huapeng
    Xu, Hui
    Zhan, Tianming
    [J]. MULTIMEDIA SYSTEMS, 2024, 30 (03)
  • [6] Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution
    Kasem, Hossam M.
    Hung, Kwok-Wai
    Jiang, Jianmin
    [J]. IEEE ACCESS, 2019, 7 : 182993 - 183009
  • [7] Dual Aggregation Transformer for Image Super-Resolution
    Chen, Zheng
    Zhang, Yulun
    Gu, Jinjin
    Kong, Linghe
    Yang, Xiaokang
    Yu, Fisher
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 12278 - 12287
  • [8] SSAformer: Spatial-Spectral Aggregation Transformer for Hyperspectral Image Super-Resolution
    Wang, Haoqian
    Zhang, Qi
    Peng, Tao
    Xu, Zhongjie
    Cheng, Xiangai
    Xing, Zhongyang
    Li, Teng
    [J]. REMOTE SENSING, 2024, 16 (10)
  • [9] A Very Deep Spatial Transformer Towards Robust Single Image Super-Resolution
    Jiang, Jianmin
    Kasem, Hossam M.
    Hung, Kwok-Wai
    [J]. IEEE ACCESS, 2019, 7 : 45618 - 45631
  • [10] Steformer: Efficient Stereo Image Super-Resolution With Transformer
    Lin, Jianxin
    Yin, Lianying
    Wang, Yijun
    [J]. IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 8396 - 8407