Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving

被引:5
|
作者
Zhang, Xiucai [1 ]
He, Lei [1 ]
Chen, Junyi [1 ]
Wang, Baoyun [1 ]
Wang, Yuhai [1 ]
Zhou, Yuanle [1 ]
机构
[1] Jilin Univ, State Key Lab Automot Simulat & Control, Changchun 130022, Peoples R China
关键词
multimodal fusion; attention mechanism; 3D target detection; deep learning; REPRESENTATION;
D O I
10.3390/s23218732
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
This paper proposes a multimodal fusion 3D target detection algorithm based on the attention mechanism to improve the performance of 3D target detection. The algorithm utilizes point cloud data and information from the camera. For image feature extraction, the ResNet50 + FPN architecture extracts features at four levels. Point cloud feature extraction employs the voxel method and FCN to extract point and voxel features. The fusion of image and point cloud features is achieved through regional point fusion and voxel fusion methods. After information fusion, the Coordinate and SimAM attention mechanisms extract fusion features at a deep level. The algorithm's performance is evaluated using the DAIR-V2X dataset. The results show that compared to the Part-A2 algorithm; the proposed algorithm improves the mAP value by 7.9% in the BEV view and 7.8% in the 3D view at IOU = 0.5 (cars) and IOU = 0.25 (pedestrians and cyclists). At IOU = 0.7 (cars) and IOU = 0.5 (pedestrians and cyclists), the mAP value of the SECOND algorithm is improved by 5.4% in the BEV view and 4.3% in the 3D view, compared to other comparison algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [31] Improved 3D Semantic Segmentation Model Based on RGB Image and LiDAR Point Cloud Fusion for Automantic Driving
    Du, Jiahao
    Huang, Xiaoci
    Xing, Mengyang
    Zhang, Tao
    INTERNATIONAL JOURNAL OF AUTOMOTIVE TECHNOLOGY, 2023, 24 (03) : 787 - 797
  • [32] 3D object detection algorithm based on multi-sensor segmental fusion of frustum association for autonomous driving
    Tao, Chongben
    Bian, Weitao
    Wang, Chen
    Li, Huayi
    Gao, Zhen
    Zhang, Zufeng
    Zheng, Sifa
    Zhu, Yuan
    APPLIED INTELLIGENCE, 2023, 53 (19) : 22753 - 22774
  • [33] Improved 3D Semantic Segmentation Model Based on RGB Image and LiDAR Point Cloud Fusion for Automantic Driving
    Jiahao Du
    Xiaoci Huang
    Mengyang Xing
    Tao Zhang
    International Journal of Automotive Technology, 2023, 24 : 787 - 797
  • [34] 3D object detection algorithm based on multi-sensor segmental fusion of frustum association for autonomous driving
    Chongben Tao
    Weitao Bian
    Chen Wang
    Huayi Li
    Zhen Gao
    Zufeng Zhang
    Sifa Zheng
    Yuan Zhu
    Applied Intelligence, 2023, 53 : 22753 - 22774
  • [35] RGB Image- and Lidar-Based 3D Object Detection Under Multiple Lighting Scenarios
    Wentao Chen
    Wei Tian
    Xiang Xie
    Wilhelm Stork
    Automotive Innovation, 2022, 5 : 251 - 259
  • [36] RGB Image- and Lidar-Based 3D Object Detection Under Multiple Lighting Scenarios
    Chen, Wentao
    Tian, Wei
    Xie, Xiang
    Stork, Wilhelm
    AUTOMOTIVE INNOVATION, 2022, 5 (03) : 251 - 259
  • [37] LiDAR 3D Object Detection Based on Improved PointRCNN
    Gao, Han
    Chen, Ying
    Ni, Lizheng
    Deng, Xiuhan
    Zhong, Kai
    Yan, Chengzhi
    LASER & OPTOELECTRONICS PROGRESS, 2024, 61 (22)
  • [38] SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detection
    Qin, Yiran
    Wang, Chaoqun
    Kang, Zijian
    Ma, Ningning
    Li, Zhen
    Zhang, Ruimao
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 21957 - 21967
  • [39] 3D Object Detection With Multi-Frame RGB-Lidar Feature Alignment
    Ercelik, Emec
    Yurtsever, Ekim
    Knoll, Alois
    IEEE ACCESS, 2021, 9 : 143138 - 143149
  • [40] FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving
    Zhang, Lei
    Li, Xu
    Tang, Kaichen
    Jiang, Yunzhe
    Yang, Liu
    Zhang, Yonggang
    Chen, Xianyi
    IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2023, 24 (11) : 12154 - 12165