MMYFnet: Multi-Modality YOLO Fusion Network for Object Detection in Remote Sensing Images

被引:0
|
作者
Guo, Huinan [1 ]
Sun, Congying [1 ,2 ]
Zhang, Jing [2 ]
Zhang, Wuxia [3 ]
Zhang, Nengshuang [1 ,2 ]
机构
[1] Chinese Acad Sci, Xian Inst Opt & Fine Mech, Xian 710119, Peoples R China
[2] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[3] Xian Univ Posts & Telecommun, Sch Comp Sci & Technol, Xian 710121, Peoples R China
基金
中国国家自然科学基金;
关键词
cross-modality; cosine similarity; feature fusion; multi-spectral remote sensing imagery; dual-branch; object detection; SIMILARITY;
D O I
10.3390/rs16234451
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Object detection in remote sensing images is crucial for airport management, hazard prevention, traffic monitoring, and more. The precise ability for object localization and identification enables remote sensing imagery to provide early warnings, mitigate risks, and offer strong support for decision-making processes. While traditional deep learning-based object detection techniques have achieved significant results in single-modal environments, their detection capabilities still encounter challenges when confronted with complex environments, such as adverse weather conditions or situations where objects are obscured. To overcome the limitations of existing fusion methods in terms of complexity and insufficient information utilization, we innovatively propose a Cosine Similarity-based Image Feature Fusion (CSIFF) module and integrate it into a dual-branch YOLOv8 network, constructing a lightweight and efficient target detection network called Multi-Modality YOLO Fusion Network (MMYFNet). This network utilizes cosine similarity to divide the original features into common features and specific features, which are then refined and fused through specific modules. Experimental and analytical results show that MMYFNet performs excellently on both the VEDAI and FLIR datasets, achieving mAP values of 80% and 76.8%, respectively. Further validation through parameter sensitivity experiments, ablation studies, and visual analyses confirms the effectiveness of the CSIFF module. MMYFNet achieves high detection accuracy with fewer parameters, and the CSIFF module, as a plug-and-play module, can be integrated into other CNN-based cross-modality network models, providing a new approach for object detection in remote sensing image fusion.
引用
收藏
页数:19
相关论文
共 50 条
  • [41] DAG-YOLO: A Context-feature Adaptive Fusion Rotating Detection Network in Remote Sensing Images
    Guo, Zhenjiang
    He, Xiaohai
    Yang, Yu
    Qing, Linbo
    Chen, Honggang
    ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (10)
  • [42] A Multi-Branch Feature Fusion Network for Building Detection in Remote Sensing Images
    Li, Chao
    Huang, Xinyu
    Tang, Jiechen
    Wang, Kai
    IEEE ACCESS, 2021, 9 (09): : 168511 - 168519
  • [43] Multi-modality hierarchical fusion network for lumbar spine segmentation with magnetic resonance images
    Yan, Han
    Zhang, Guangtao
    Cui, Wei
    Yu, Zhuliang
    CONTROL THEORY AND TECHNOLOGY, 2024, 22 (04) : 612 - 622
  • [44] Application of the deep fusion mechanism in object detection of remote sensing images
    Dong R.
    Jiao L.
    Zhao J.
    Shen W.
    Xi'an Dianzi Keji Daxue Xuebao/Journal of Xidian University, 2021, 48 (05): : 128 - 138
  • [45] Multi-source information fusion attention network for weakly supervised salient object detection in optical remote sensing images
    Yan, Longquan
    Yang, Shuhui
    Zhang, Qi
    Yan, Ruixiang
    Wang, Tao
    Liu, Hengzhi
    Zhou, Mingquan
    EXPERT SYSTEMS WITH APPLICATIONS, 2025, 261
  • [46] YOLO-Remote: An Object Detection Algorithm for Remote Sensing Targets
    Fan, Kaizhe
    Li, Qian
    Li, Quanjun
    Zhong, Guangqi
    Chu, Yue
    Le, Zhen
    Xu, Yeling
    Li, Jianfeng
    IEEE ACCESS, 2024, 12 : 155654 - 155665
  • [47] A MULTI-BRANCH U-NET FOR WATER AREA SEGMENTATION WITH MULTI-MODALITY REMOTE SENSING IMAGES
    Zhang, Chenchen
    Wang, Rongfang
    Chen, Jia-Wei
    Li, Weibin
    Huo, Chunlei
    Niu, Yi
    IGARSS 2023 - 2023 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, 2023, : 5443 - 5446
  • [48] Multi-Content Complementation Network for Salient Object Detection in Optical Remote Sensing Images
    Li, Gongyang
    Liu, Zhi
    Lin, Weisi
    Ling, Haibin
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
  • [49] Multi-Oriented Rotation-Equivariant Network for Object Detection on Remote Sensing Images
    Zhu, Kun
    Zhang, Xiaodong
    Chen, Guanzhou
    Li, Xianwei
    Cai, Peihua
    Liao, Puyun
    Wang, Tong
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2022, 19
  • [50] Anchor-Free Network for Multi-class Object Detection in Remote Sensing Images
    Zhao, Guochuan
    Pang, Jie
    Zhang, Hua
    Zhou, Jian
    Li, Linjing
    PROCEEDINGS OF THE 39TH CHINESE CONTROL CONFERENCE, 2020, : 7510 - 7515