A Multiscale Spatiotemporal Fusion Network Based on an Attention Mechanism

被引:4
|
作者
Huang, Zhiqiang [1 ,2 ]
Li, Yujia [2 ,3 ]
Bai, Menghao [1 ,2 ]
Wei, Qing [1 ]
Gu, Qian [1 ]
Mou, Zhijun [1 ]
Zhang, Liping [2 ]
Lei, Dajiang [1 ,2 ]
机构
[1] Chongqing Univ Posts & Telecommun, Coll Comp, Chongqing 400065, Peoples R China
[2] Chongqing Univ Posts & Telecommun, Chongqing Key Lab Image Cognit, Chongqing 400065, Peoples R China
[3] Chongqing Univ Posts & Telecommun, Sch Cyber Secur & Informat Law, Chongqing 400065, Peoples R China
基金
中国国家自然科学基金;
关键词
spatiotemporal fusion; multiscale feature fusion; attention mechanism; compound loss function; CONVOLUTIONAL NEURAL-NETWORK; MODIS; LANDSAT; IMAGES;
D O I
10.3390/rs15010182
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Spatiotemporal fusion is an effective and cost-effective method to obtain both high temporal resolution and high spatial resolution images. However, existing methods do not sufficiently extract the deeper features of the image, resulting in fused images which do not recover good topographic detail and poor fusion quality. In order to obtain higher quality spatiotemporal fusion images, a novel spatiotemporal fusion method based on deep learning is proposed in this paper. The method combines an attention mechanism and a multiscale feature fusion network to design a network that more scientifically explores deeper features of the image for different input image characteristics. Specifically, a multiscale feature fusion module is introduced into the spatiotemporal fusion task and combined with an efficient spatial-channel attention module to improve the capture of spatial and channel information while obtaining more effective information. In addition, we design a new edge loss function and incorporate it into the compound loss function, which helps to generate fused images with richer edge information. In terms of both index performance and image details, our proposed model has excellent results on both datasets compared with the current mainstream spatiotemporal fusion methods.
引用
收藏
页数:18
相关论文
共 50 条
  • [41] Fusion network for small target detection based on YOLO and attention mechanism
    XU Caie
    DONG Zhe
    ZHONG Shengyun
    CHEN Yijiang
    PAN Sishun
    WU Mingyang
    Optoelectronics Letters, 2024, 20 (06) : 372 - 378
  • [42] Point Cloud Registration Network Based on Convolution Fusion and Attention Mechanism
    Wei Zhu
    Yue Ying
    Jin Zhang
    Xiuli Wang
    Yayu Zheng
    Neural Processing Letters, 2023, 55 : 12625 - 12645
  • [43] Point Cloud Registration Network Based on Convolution Fusion and Attention Mechanism
    Zhu, Wei
    Ying, Yue
    Zhang, Jin
    Wang, Xiuli
    Zheng, Yayu
    NEURAL PROCESSING LETTERS, 2023, 55 (09) : 12625 - 12645
  • [44] Multimodal Sentiment Analysis Based on Attention Mechanism and Tensor Fusion Network
    Zhang, Kang
    Geng, Yushui
    Zhao, Jing
    Li, Wenxiao
    Liu, Jianxin
    2021 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS (SMC), 2021, : 1473 - 1477
  • [45] Fusion network for small target detection based on YOLO and attention mechanism
    Xu, Caie
    Dong, Zhe
    Zhong, Shengyun
    Chen, Yijiang
    Pan, Sishun
    Wu, Mingyang
    OPTOELECTRONICS LETTERS, 2024, 20 (06) : 372 - 378
  • [46] A Multiscale Spatiotemporal Attention Network for Ground-Based Remote Sensing Cloud Image Sequence Prediction
    Zhang, Feng
    Cheng, Yingying
    Hua, Qiang
    Dong, Chunru
    Zhang, Yong
    Wu, Tingdong
    IEEE Transactions on Geoscience and Remote Sensing, 2024, 62
  • [47] Graphic image classification method based on an attention mechanism and fusion of multilevel and multiscale deep features
    Liu, Shan
    Zhang, Qi
    Huang, Lingling
    COMPUTER COMMUNICATIONS, 2023, 209 : 230 - 238
  • [48] Fault Diagnosis Method for EPR Cable Terminals Based on Attention Mechanism and Multiscale Network
    融合注意力机制与多尺度网络的EPR电缆终端故障诊断方法
    Zhou, Lijun (ljzhou10@163.com), 1600, Science Press (47): : 3872 - 3880
  • [49] Video Dehazing Network Based on Multiscale Attention
    Han, Congwei
    Zhang, Kangkang
    Brekhna, Brekhna
    IEEE ACCESS, 2023, 11 : 94479 - 94485
  • [50] Optimization of the routing of capsule network based on multiscale information and self-attention mechanism
    Shang, Yunhao
    Xu, Ning
    Jin, Zhenzhou
    Yao, Xiao
    JOURNAL OF ELECTRONIC IMAGING, 2022, 31 (02)