Adaptive learning point cloud and image diversity feature fusion network for 3D object detection

被引:0
|
作者
Weiqing Yan
Shile Liu
Hao Liu
Guanghui Yue
Xuan Wang
Yongchao Song
Jindong Xu
机构
[1] Yantai University,School of Computer and Control Engineering
[2] Shenzhen University,School of Biomedical Engineering, Health Science Center
来源
关键词
3D object detection; LiDAR point cloud; Fine-grained image; Diversity feature fusion;
D O I
暂无
中图分类号
学科分类号
摘要
3D object detection is a critical task in the fields of virtual reality and autonomous driving. Given that each sensor has its own strengths and limitations, multi-sensor-based 3D object detection has gained popularity. However, most existing methods extract high-level image semantic features and fuse them with point cloud features, focusing solely on consistent information from both sensors while ignoring their complementary information. In this paper, we present a novel two-stage multi-sensor deep neural network, called the adaptive learning point cloud and image diversity feature fusion network (APIDFF-Net), for 3D object detection. Our approach employs the fine-grained image information to complement the point cloud information by combining low-level image features with high-level point cloud features. Specifically, we design a shallow image feature extraction module to learn fine-grained information from images, instead of relying on deep layer features with coarse-grained information. Furthermore, we design a diversity feature fusion (DFF) module that transforms low-level image features into point-wise image features and explores their complementary features through an attention mechanism, ensuring an effective combination of fine-grained image features and point cloud features. Experiments on the KITTI benchmark show that the proposed method outperforms state-of-the-art methods.
引用
收藏
页码:2825 / 2837
页数:12
相关论文
共 50 条
  • [41] PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection
    Chen, Anthony
    Zhang, Kevin
    Zhang, Renrui
    Wang, Zihan
    Lu, Yuheng
    Guo, Yandong
    Zhang, Shanghang
    2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR, 2023, : 5291 - 5301
  • [42] Multi-View Joint Learning and BEV Feature-Fusion Network for 3D Object Detection
    Liu, Qunming
    Li, Xiaodong
    Zhang, Xiaofei
    Tan, Xiaojun
    Shi, Bodong
    APPLIED SCIENCES-BASEL, 2023, 13 (09):
  • [43] A novel multi-model 3D object detection framework with adaptive voxel-image feature fusion
    Liu, Zhao
    Fu, Zhongliang
    Li, Gang
    Zhang, Shengyuan
    IET COMPUTER VISION, 2024, 18 (05) : 640 - 651
  • [44] MFF-Net: Multimodal Feature Fusion Network for 3D Object Detection
    Shi, Peicheng
    Liu, Zhiqiang
    Qi, Heng
    Yang, Aixi
    CMC-COMPUTERS MATERIALS & CONTINUA, 2023, 75 (03): : 5615 - 5637
  • [45] Deformable Feature Fusion Network for Multi-Modal 3D Object Detection
    Guo, Kun
    Gan, Tong
    Ding, Zhao
    Ling, Qiang
    2024 3RD INTERNATIONAL CONFERENCE ON ROBOTICS, ARTIFICIAL INTELLIGENCE AND INTELLIGENT CONTROL, RAIIC 2024, 2024, : 363 - 367
  • [46] PSANet: Pyramid Splitting and Aggregation Network for 3D Object Detection in Point Cloud
    Li, Fangyu
    Jin, Weizheng
    Fan, Cien
    Zou, Lian
    Chen, Qingsheng
    Li, Xiaopeng
    Jiang, Hao
    Liu, Yifeng
    SENSORS, 2021, 21 (01) : 1 - 21
  • [47] Spatial information enhancement network for 3D object detection from point cloud
    Li, Ziyu
    Yao, Yuncong
    Quan, Zhibin
    Xie, Jin
    Yang, Wankou
    PATTERN RECOGNITION, 2022, 128
  • [48] SCNet: Subdivision Coding Network for Object Detection Based on 3D Point Cloud
    Wang, Zhiyu
    Fu, Hao
    Wang, Li
    Xiao, Liang
    Dai, Bin
    IEEE ACCESS, 2019, 7 : 120449 - 120462
  • [49] Multimodal 3D Object Detection Method Based on Pseudo Point Cloud Feature Enhancement
    Kong D.-M.
    Li X.-W.
    Yang Q.-X.
    Jisuanji Xuebao/Chinese Journal of Computers, 2024, 47 (04): : 759 - 775
  • [50] 3D Point Cloud Semantic Segmentation Network Based on Coding Feature Learning
    Tong, Guofeng
    Liu, Yongxu
    Peng, Hao
    Shao, Yuyuan
    Moshi Shibie yu Rengong Zhineng/Pattern Recognition and Artificial Intelligence, 2023, 36 (04): : 313 - 326