MMYFnet: Multi-Modality YOLO Fusion Network for Object Detection in Remote Sensing Images

被引:0
|
作者
Guo, Huinan [1 ]
Sun, Congying [1 ,2 ]
Zhang, Jing [2 ]
Zhang, Wuxia [3 ]
Zhang, Nengshuang [1 ,2 ]
机构
[1] Chinese Acad Sci, Xian Inst Opt & Fine Mech, Xian 710119, Peoples R China
[2] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[3] Xian Univ Posts & Telecommun, Sch Comp Sci & Technol, Xian 710121, Peoples R China
基金
中国国家自然科学基金;
关键词
cross-modality; cosine similarity; feature fusion; multi-spectral remote sensing imagery; dual-branch; object detection; SIMILARITY;
D O I
10.3390/rs16234451
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Object detection in remote sensing images is crucial for airport management, hazard prevention, traffic monitoring, and more. The precise ability for object localization and identification enables remote sensing imagery to provide early warnings, mitigate risks, and offer strong support for decision-making processes. While traditional deep learning-based object detection techniques have achieved significant results in single-modal environments, their detection capabilities still encounter challenges when confronted with complex environments, such as adverse weather conditions or situations where objects are obscured. To overcome the limitations of existing fusion methods in terms of complexity and insufficient information utilization, we innovatively propose a Cosine Similarity-based Image Feature Fusion (CSIFF) module and integrate it into a dual-branch YOLOv8 network, constructing a lightweight and efficient target detection network called Multi-Modality YOLO Fusion Network (MMYFNet). This network utilizes cosine similarity to divide the original features into common features and specific features, which are then refined and fused through specific modules. Experimental and analytical results show that MMYFNet performs excellently on both the VEDAI and FLIR datasets, achieving mAP values of 80% and 76.8%, respectively. Further validation through parameter sensitivity experiments, ablation studies, and visual analyses confirms the effectiveness of the CSIFF module. MMYFNet achieves high detection accuracy with fewer parameters, and the CSIFF module, as a plug-and-play module, can be integrated into other CNN-based cross-modality network models, providing a new approach for object detection in remote sensing image fusion.
引用
收藏
页数:19
相关论文
共 50 条
  • [31] Multi-modality Fusion Network for Action Recognition
    Huang, Kai
    Qin, Zheng
    Xu, Kaiping
    Ye, Shuxiong
    Wang, Guolong
    ADVANCES IN MULTIMEDIA INFORMATION PROCESSING - PCM 2017, PT II, 2018, 10736 : 139 - 149
  • [32] Dynamic Feature Fusion for Object Detection in Remote Sensing Images
    Xie, Xing-Xing
    Cheng, Gong
    Yao, Yan-Qing
    Yao, Xi-Wen
    Han, Jun-Wei
    Jisuanji Xuebao/Chinese Journal of Computers, 2022, 45 (04): : 735 - 747
  • [33] Fusion Of Multi-Modality Medical Images: A Fuzzy Approach
    Kaur, Harmeet
    Kumar, Satish
    PROCEEDINGS ON 2018 IEEE 3RD INTERNATIONAL CONFERENCE ON COMPUTING, COMMUNICATION AND SECURITY (ICCCS), 2018, : 112 - 115
  • [34] Automated registration and fusion of the multi-modality retinal images
    Cao, Hua
    Brener, Nathan
    Thompson, Hilary
    Iyengar, S. S.
    Ye, Zhengmao
    PROCEEDINGS OF THE 40TH SOUTHEASTERN SYMPOSIUM ON SYSTEM THEORY, 2008, : 371 - +
  • [35] ACDF-YOLO: Attentive and Cross-Differential Fusion Network for Multimodal Remote Sensing Object Detection
    Fei, Xuan
    Guo, Mengyao
    Li, Yan
    Yu, Renping
    Sun, Le
    REMOTE SENSING, 2024, 16 (18)
  • [36] YOLO-MFD: Remote Sensing Image Object Detection with Multi-Scale Fusion Dynamic Head
    Zhang, Zhongyuan
    Zhu, Wenqiu
    CMC-COMPUTERS MATERIALS & CONTINUA, 2024, 79 (02): : 2547 - 2563
  • [37] A multi-scale small object detection algorithm SMA-YOLO for UAV remote sensing images
    Zhou, Shilong
    Zhou, Haijin
    Qian, Lei
    SCIENTIFIC REPORTS, 2025, 15 (01):
  • [38] MRFF-YOLO: A Multi-Receptive Fields Fusion Network for Remote Sensing Target Detection
    Xu, Danqing
    Wu, Yiquan
    REMOTE SENSING, 2020, 12 (19)
  • [39] Dilated Convolution and Feature Fusion SSD Network for Small Object Detection in Remote Sensing Images
    Qu, Junsuo
    Su, Chang
    Zhang, Zhiwei
    Razi, Abolfazl
    IEEE ACCESS, 2020, 8 : 82832 - 82843
  • [40] A Task-Balanced Multiscale Adaptive Fusion Network for Object Detection in Remote Sensing Images
    Gao, Tao
    Liu, Zixiang
    Zhang, Jing
    Wu, Guiping
    Chen, Ting
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61