MMYFnet: Multi-Modality YOLO Fusion Network for Object Detection in Remote Sensing Images

被引:0
|
作者
Guo, Huinan [1 ]
Sun, Congying [1 ,2 ]
Zhang, Jing [2 ]
Zhang, Wuxia [3 ]
Zhang, Nengshuang [1 ,2 ]
机构
[1] Chinese Acad Sci, Xian Inst Opt & Fine Mech, Xian 710119, Peoples R China
[2] Xian Univ Technol, Sch Automat & Informat Engn, Xian 710048, Peoples R China
[3] Xian Univ Posts & Telecommun, Sch Comp Sci & Technol, Xian 710121, Peoples R China
基金
中国国家自然科学基金;
关键词
cross-modality; cosine similarity; feature fusion; multi-spectral remote sensing imagery; dual-branch; object detection; SIMILARITY;
D O I
10.3390/rs16234451
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Object detection in remote sensing images is crucial for airport management, hazard prevention, traffic monitoring, and more. The precise ability for object localization and identification enables remote sensing imagery to provide early warnings, mitigate risks, and offer strong support for decision-making processes. While traditional deep learning-based object detection techniques have achieved significant results in single-modal environments, their detection capabilities still encounter challenges when confronted with complex environments, such as adverse weather conditions or situations where objects are obscured. To overcome the limitations of existing fusion methods in terms of complexity and insufficient information utilization, we innovatively propose a Cosine Similarity-based Image Feature Fusion (CSIFF) module and integrate it into a dual-branch YOLOv8 network, constructing a lightweight and efficient target detection network called Multi-Modality YOLO Fusion Network (MMYFNet). This network utilizes cosine similarity to divide the original features into common features and specific features, which are then refined and fused through specific modules. Experimental and analytical results show that MMYFNet performs excellently on both the VEDAI and FLIR datasets, achieving mAP values of 80% and 76.8%, respectively. Further validation through parameter sensitivity experiments, ablation studies, and visual analyses confirms the effectiveness of the CSIFF module. MMYFNet achieves high detection accuracy with fewer parameters, and the CSIFF module, as a plug-and-play module, can be integrated into other CNN-based cross-modality network models, providing a new approach for object detection in remote sensing image fusion.
引用
收藏
页数:19
相关论文
共 50 条
  • [21] RSI-YOLO: Object Detection Method for Remote Sensing Images Based on Improved YOLO
    Li, Zhuang
    Yuan, Jianhui
    Li, Guixiang
    Wang, Hao
    Li, Xingcan
    Li, Dan
    Wang, Xinhua
    SENSORS, 2023, 23 (14)
  • [22] A Multiscale Information Fusion Network Based on PixelShuffle Integrated With YOLO for Aerial Remote Sensing Object Detection
    Xi, Li Hu
    Hou, Jing Wei
    Ma, Guang Lin
    Hei, Yong Qiang
    Li, Wen Tao
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2024, 21
  • [23] C2-YOLO: Rotating Object Detection Network for Remote Sensing Images with Complex Backgrounds
    Cheng, Xiaotong
    Zhang, Chongyang
    2022 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2022,
  • [24] Lightweight Multi-Scale Feature Fusion Network for Salient Object Detection in Optical Remote Sensing Images
    Li, Jun
    Huang, Kaigen
    ELECTRONICS, 2025, 14 (01):
  • [25] MSNet: Multi-Scale Network for Object Detection in Remote Sensing Images
    Gao, Tao
    Xia, Shilin
    Liu, Mengkun
    Zhang, Jing
    Chen, Ting
    Li, Ziqi
    PATTERN RECOGNITION, 2025, 158
  • [26] An Adaptive Attention Fusion Mechanism Convolutional Network for Object Detection in Remote Sensing Images
    Ye, Yuanxin
    Ren, Xiaoyue
    Zhu, Bai
    Tang, Tengfeng
    Tan, Xin
    Gui, Yang
    Yao, Qin
    REMOTE SENSING, 2022, 14 (03)
  • [27] CMR-net: A cross modality reconstruction network for multi-modality remote sensing classification
    Wang, Huiqing
    Wang, Huajun
    Wu, Lingfeng
    PLOS ONE, 2024, 19 (06):
  • [28] SCM-YOLO for Lightweight Small Object Detection in Remote Sensing Images
    Qiang, Hao
    Hao, Wei
    Xie, Meilin
    Tang, Qiang
    Shi, Heng
    Zhao, Yixin
    Han, Xiaoteng
    REMOTE SENSING, 2025, 17 (02)
  • [29] Lightweight Progressive Fusion Calibration Network for Rotated Object Detection in Remote Sensing Images
    Liu, Jing
    Jing, Donglin
    Cao, Yanyan
    Wang, Ying
    Guo, Chaoping
    Shi, Peijun
    Zhang, Haijing
    ELECTRONICS, 2024, 13 (16)
  • [30] General Optimization Methods for YOLO Series Object Detection in Remote Sensing Images
    Nan, Guozheng
    Zhao, Yue
    Lin, Chengxing
    Ye, Qiaolin
    IEEE SIGNAL PROCESSING LETTERS, 2024, 31 : 2860 - 2864