A Multiscale Spatiotemporal Fusion Network Based on an Attention Mechanism

被引:4
|
作者
Huang, Zhiqiang [1 ,2 ]
Li, Yujia [2 ,3 ]
Bai, Menghao [1 ,2 ]
Wei, Qing [1 ]
Gu, Qian [1 ]
Mou, Zhijun [1 ]
Zhang, Liping [2 ]
Lei, Dajiang [1 ,2 ]
机构
[1] Chongqing Univ Posts & Telecommun, Coll Comp, Chongqing 400065, Peoples R China
[2] Chongqing Univ Posts & Telecommun, Chongqing Key Lab Image Cognit, Chongqing 400065, Peoples R China
[3] Chongqing Univ Posts & Telecommun, Sch Cyber Secur & Informat Law, Chongqing 400065, Peoples R China
基金
中国国家自然科学基金;
关键词
spatiotemporal fusion; multiscale feature fusion; attention mechanism; compound loss function; CONVOLUTIONAL NEURAL-NETWORK; MODIS; LANDSAT; IMAGES;
D O I
10.3390/rs15010182
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Spatiotemporal fusion is an effective and cost-effective method to obtain both high temporal resolution and high spatial resolution images. However, existing methods do not sufficiently extract the deeper features of the image, resulting in fused images which do not recover good topographic detail and poor fusion quality. In order to obtain higher quality spatiotemporal fusion images, a novel spatiotemporal fusion method based on deep learning is proposed in this paper. The method combines an attention mechanism and a multiscale feature fusion network to design a network that more scientifically explores deeper features of the image for different input image characteristics. Specifically, a multiscale feature fusion module is introduced into the spatiotemporal fusion task and combined with an efficient spatial-channel attention module to improve the capture of spatial and channel information while obtaining more effective information. In addition, we design a new edge loss function and incorporate it into the compound loss function, which helps to generate fused images with richer edge information. In terms of both index performance and image details, our proposed model has excellent results on both datasets compared with the current mainstream spatiotemporal fusion methods.
引用
收藏
页数:18
相关论文
共 50 条
  • [1] Multiscale Attention Spatiotemporal Fusion Model Based on Pyramidal Network Constraints
    Ran, Qiong
    Wang, Qiuhui
    Zheng, Ke
    Li, Jiaxin
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2024, 21
  • [2] A Spatiotemporal Fusion Method Based on Multiscale Feature Extraction and Spatial Channel Attention Mechanism
    Lei, Dajiang
    Ran, Gangsheng
    Zhang, Liping
    Li, Weisheng
    REMOTE SENSING, 2022, 14 (03)
  • [3] MANet: A Network Architecture for Remote Sensing Spatiotemporal Fusion Based on Multiscale and Attention Mechanisms
    Cao, Huimin
    Luo, Xiaobo
    Peng, Yidong
    Xie, Tianshou
    REMOTE SENSING, 2022, 14 (18)
  • [4] Attention-Based Multiscale Spatiotemporal Network for Traffic Forecast with Fusion of External Factors
    Nadarajan, Jeba
    Sivanraj, Rathi
    ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2022, 11 (12)
  • [5] A multiscale feature fusion network based on attention mechanism for motor imagery EEG decoding
    Gao, Dongrui
    Yang, Wen
    Li, Pengrui
    Liu, Shihong
    Liu, Tiejun
    Wang, Manqing
    Zhang, Yongqing
    APPLIED SOFT COMPUTING, 2024, 151
  • [6] Emotion Recognition via Multiscale Feature Fusion Network and Attention Mechanism
    Jiang, Yiye
    Xie, Songyun
    Xie, Xinzhou
    Cui, Yujie
    Tang, Hao
    IEEE SENSORS JOURNAL, 2023, 23 (10) : 10790 - 10800
  • [7] Dynamic Gesture Recognition Network Based on Multiscale Spatiotemporal Feature Fusion
    Liu, Jie
    Wang, Yue
    Tian, Ming
    JOURNAL OF ELECTRONICS & INFORMATION TECHNOLOGY, 2023, 45 (07) : 2614 - 2622
  • [8] Spatiotemporal Fusion of Remote Sensing Images using a Convolutional Neural Network with Attention and Multiscale Mechanisms
    Li, Weisheng
    Zhang, Xiayan
    Peng, Yidong
    Dong, Meilin
    INTERNATIONAL JOURNAL OF REMOTE SENSING, 2021, 42 (06) : 1973 - 1993
  • [9] MFNet: Panoptic segmentation network based on multiscale feature weighted fusion and frequency domain attention mechanism
    Lei, Haiwei
    He, Fangyuan
    Jia, Bohui
    Wu, Qian
    IET COMPUTER VISION, 2023, 17 (01) : 88 - 97
  • [10] Remote Sensing Data Detection Based on Multiscale Fusion and Attention Mechanism
    Huang, Min
    Cheng, Cong
    De Luca, Gennaro
    MOBILE INFORMATION SYSTEMS, 2021, 2021