A Multiscale Spatiotemporal Fusion Network Based on an Attention Mechanism

被引:4
|
作者
Huang, Zhiqiang [1 ,2 ]
Li, Yujia [2 ,3 ]
Bai, Menghao [1 ,2 ]
Wei, Qing [1 ]
Gu, Qian [1 ]
Mou, Zhijun [1 ]
Zhang, Liping [2 ]
Lei, Dajiang [1 ,2 ]
机构
[1] Chongqing Univ Posts & Telecommun, Coll Comp, Chongqing 400065, Peoples R China
[2] Chongqing Univ Posts & Telecommun, Chongqing Key Lab Image Cognit, Chongqing 400065, Peoples R China
[3] Chongqing Univ Posts & Telecommun, Sch Cyber Secur & Informat Law, Chongqing 400065, Peoples R China
基金
中国国家自然科学基金;
关键词
spatiotemporal fusion; multiscale feature fusion; attention mechanism; compound loss function; CONVOLUTIONAL NEURAL-NETWORK; MODIS; LANDSAT; IMAGES;
D O I
10.3390/rs15010182
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Spatiotemporal fusion is an effective and cost-effective method to obtain both high temporal resolution and high spatial resolution images. However, existing methods do not sufficiently extract the deeper features of the image, resulting in fused images which do not recover good topographic detail and poor fusion quality. In order to obtain higher quality spatiotemporal fusion images, a novel spatiotemporal fusion method based on deep learning is proposed in this paper. The method combines an attention mechanism and a multiscale feature fusion network to design a network that more scientifically explores deeper features of the image for different input image characteristics. Specifically, a multiscale feature fusion module is introduced into the spatiotemporal fusion task and combined with an efficient spatial-channel attention module to improve the capture of spatial and channel information while obtaining more effective information. In addition, we design a new edge loss function and incorporate it into the compound loss function, which helps to generate fused images with richer edge information. In terms of both index performance and image details, our proposed model has excellent results on both datasets compared with the current mainstream spatiotemporal fusion methods.
引用
收藏
页数:18
相关论文
共 50 条
  • [31] Hyperspectral Image Classification with a Multiscale Fusion-Evolution Graph Convolutional Network Based on a Feature-Spatial Attention Mechanism
    Jing, Haoyu
    Wang, Yuanyuan
    Du, Zhenhong
    Zhang, Feng
    REMOTE SENSING, 2022, 14 (11)
  • [32] An Attention-Based Context Fusion Network for Spatiotemporal Prediction of Sea Surface Temperature
    Shi, Benyun
    Hao, Yingjian
    Feng, Liu
    Ge, Conghui
    Peng, Yue
    He, Hailun
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2024, 21
  • [33] Multiscale channel attention network for infrared and visible image fusion
    Zhu, Jiahui
    Dou, Qingyu
    Jian, Lihua
    Liu, Kai
    Hussain, Farhan
    Yang, Xiaomin
    CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2021, 33 (22):
  • [34] MAFusion: Multiscale Attention Network for Infrared and Visible Image Fusion
    Li, Xiaoling
    Chen, Houjin
    Li, Yanfeng
    Peng, Yahui
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2022, 71
  • [35] A multiscale residual pyramid attention network for medical image fusion
    Fu, Jun
    Li, Weisheng
    Du, Jiao
    Huang, Yuping
    BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2021, 66
  • [36] Shear-Wave Velocity Prediction Method via a Gate Recurrent Unit Fusion Network Based on the Spatiotemporal Attention Mechanism
    Chen, Tengfei
    Gao, Gang
    Li, Yonggen
    Wang, Peng
    Zhao, Bin
    Gui, Zhixian
    Zhai, Xiaoyan
    LITHOSPHERE, 2022, 2022 (SpecialIssue12)
  • [37] Spatiotemporal attention enhanced features fusion network for action recognition
    Danfeng Zhuang
    Min Jiang
    Jun Kong
    Tianshan Liu
    International Journal of Machine Learning and Cybernetics, 2021, 12 : 823 - 841
  • [38] Image Dehazing Based on Deep Multiscale Fusion Network and Continuous Memory Mechanism
    Li, Qiang
    Xie, Zhihua
    Zong, Sha
    Liu, Guodong
    INTELLIGENT COMPUTING METHODOLOGIES, PT III, 2022, 13395 : 409 - 422
  • [39] A Heterogeneous Spatiotemporal Attention Fusion Prediction Network for Precipitation Nowcasting
    Niu, Dan
    Che, Hongshu
    Shi, Chunlei
    Zang, Zengliang
    Wang, Hongbin
    Chen, Xunlai
    Huang, Qunbo
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2023, 16 : 8286 - 8296
  • [40] Spatiotemporal attention enhanced features fusion network for action recognition
    Zhuang, Danfeng
    Jiang, Min
    Kong, Jun
    Liu, Tianshan
    INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2021, 12 (03) : 823 - 841