Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving

被引:5
|
作者
Zhang, Xiucai [1 ]
He, Lei [1 ]
Chen, Junyi [1 ]
Wang, Baoyun [1 ]
Wang, Yuhai [1 ]
Zhou, Yuanle [1 ]
机构
[1] Jilin Univ, State Key Lab Automot Simulat & Control, Changchun 130022, Peoples R China
关键词
multimodal fusion; attention mechanism; 3D target detection; deep learning; REPRESENTATION;
D O I
10.3390/s23218732
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
This paper proposes a multimodal fusion 3D target detection algorithm based on the attention mechanism to improve the performance of 3D target detection. The algorithm utilizes point cloud data and information from the camera. For image feature extraction, the ResNet50 + FPN architecture extracts features at four levels. Point cloud feature extraction employs the voxel method and FCN to extract point and voxel features. The fusion of image and point cloud features is achieved through regional point fusion and voxel fusion methods. After information fusion, the Coordinate and SimAM attention mechanisms extract fusion features at a deep level. The algorithm's performance is evaluated using the DAIR-V2X dataset. The results show that compared to the Part-A2 algorithm; the proposed algorithm improves the mAP value by 7.9% in the BEV view and 7.8% in the 3D view at IOU = 0.5 (cars) and IOU = 0.25 (pedestrians and cyclists). At IOU = 0.7 (cars) and IOU = 0.5 (pedestrians and cyclists), the mAP value of the SECOND algorithm is improved by 5.4% in the BEV view and 4.3% in the 3D view, compared to other comparison algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] 3D object detection based on image and LIDAR fusion for autonomous driving
    Chen G.
    Yi H.
    Mao Z.
    International Journal of Vehicle Information and Communication Systems, 2023, 8 (03) : 237 - 251
  • [2] RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving
    El Madawi, Khaled
    Rashed, Hazem
    El Sallab, Ahmad
    Nasr, Omar
    Kamel, Hanan
    Yogamani, Senthil
    2019 IEEE INTELLIGENT TRANSPORTATION SYSTEMS CONFERENCE (ITSC), 2019, : 7 - 12
  • [3] Stereo RGB and Deeper LIDAR-Based Network for 3D Object Detection in Autonomous Driving
    He, Qingdong
    Wang, Zhengning
    Zeng, Hao
    Zeng, Yi
    Liu, Yijun
    Liu, Shuaicheng
    Zeng, Bing
    IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2023, 24 (01) : 152 - 162
  • [4] RGB-LiDAR fusion for accurate 2D and 3D object detection
    Mousa-Pasandi, Morteza
    Liu, Tianran
    Massoud, Yahya
    Laganiere, Robert
    MACHINE VISION AND APPLICATIONS, 2023, 34 (05)
  • [5] RGB-LiDAR fusion for accurate 2D and 3D object detection
    Morteza Mousa-Pasandi
    Tianran Liu
    Yahya Massoud
    Robert Laganière
    Machine Vision and Applications, 2023, 34
  • [6] A LiDAR-Camera Fusion 3D Object Detection Algorithm
    Liu, Leyuan
    He, Jian
    Ren, Keyan
    Xiao, Zhonghua
    Hou, Yibin
    INFORMATION, 2022, 13 (04)
  • [7] Reinforcing LiDAR-Based 3D Object Detection with RGB and 3D Information
    Liu, Wenjian
    Zhou, Yue
    NEURAL INFORMATION PROCESSING (ICONIP 2019), PT II, 2019, 11954 : 199 - 209
  • [8] LiDAR-based 3D Object Detection for Autonomous Driving
    Li, Zirui
    2022 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, COMPUTER VISION AND MACHINE LEARNING (ICICML), 2022, : 507 - 512
  • [9] Fusion of an RGB camera and LiDAR sensor through a Graph CNN for 3D object detection
    Choi, Jinsol
    Shin, Minwoo
    Paik, Joonki
    OPTICS CONTINUUM, 2023, 2 (05): : 1166 - 1179
  • [10] BEV Space 3D Object Detection Algorithm Based on Fusion of Infrared Camera and LiDAR
    Wang Wuyue
    Xu Zhaofei
    Qu Chunyan
    Lin Ying
    Chen Yufeng
    Liao Jian
    ACTA PHOTONICA SINICA, 2024, 53 (01)