Adaptive learning point cloud and image diversity feature fusion network for 3D object detection

被引:0
|
作者
Weiqing Yan
Shile Liu
Hao Liu
Guanghui Yue
Xuan Wang
Yongchao Song
Jindong Xu
机构
[1] Yantai University,School of Computer and Control Engineering
[2] Shenzhen University,School of Biomedical Engineering, Health Science Center
来源
关键词
3D object detection; LiDAR point cloud; Fine-grained image; Diversity feature fusion;
D O I
暂无
中图分类号
学科分类号
摘要
3D object detection is a critical task in the fields of virtual reality and autonomous driving. Given that each sensor has its own strengths and limitations, multi-sensor-based 3D object detection has gained popularity. However, most existing methods extract high-level image semantic features and fuse them with point cloud features, focusing solely on consistent information from both sensors while ignoring their complementary information. In this paper, we present a novel two-stage multi-sensor deep neural network, called the adaptive learning point cloud and image diversity feature fusion network (APIDFF-Net), for 3D object detection. Our approach employs the fine-grained image information to complement the point cloud information by combining low-level image features with high-level point cloud features. Specifically, we design a shallow image feature extraction module to learn fine-grained information from images, instead of relying on deep layer features with coarse-grained information. Furthermore, we design a diversity feature fusion (DFF) module that transforms low-level image features into point-wise image features and explores their complementary features through an attention mechanism, ensuring an effective combination of fine-grained image features and point cloud features. Experiments on the KITTI benchmark show that the proposed method outperforms state-of-the-art methods.
引用
收藏
页码:2825 / 2837
页数:12
相关论文
共 50 条
  • [41] A novel multi-model 3D object detection framework with adaptive voxel-image feature fusion
    Liu, Zhao
    Fu, Zhongliang
    Li, Gang
    Zhang, Shengyuan
    [J]. IET COMPUTER VISION, 2024, 18 (05) : 640 - 651
  • [42] MFF-Net: Multimodal Feature Fusion Network for 3D Object Detection
    Shi, Peicheng
    Liu, Zhiqiang
    Qi, Heng
    Yang, Aixi
    [J]. CMC-COMPUTERS MATERIALS & CONTINUA, 2023, 75 (03): : 5615 - 5637
  • [43] PSANet: Pyramid Splitting and Aggregation Network for 3D Object Detection in Point Cloud
    Li, Fangyu
    Jin, Weizheng
    Fan, Cien
    Zou, Lian
    Chen, Qingsheng
    Li, Xiaopeng
    Jiang, Hao
    Liu, Yifeng
    [J]. SENSORS, 2021, 21 (01) : 1 - 21
  • [44] Spatial information enhancement network for 3D object detection from point cloud
    Li, Ziyu
    Yao, Yuncong
    Quan, Zhibin
    Xie, Jin
    Yang, Wankou
    [J]. PATTERN RECOGNITION, 2022, 128
  • [45] SCNet: Subdivision Coding Network for Object Detection Based on 3D Point Cloud
    Wang, Zhiyu
    Fu, Hao
    Wang, Li
    Xiao, Liang
    Dai, Bin
    [J]. IEEE ACCESS, 2019, 7 : 120449 - 120462
  • [46] Multimodal 3D Object Detection Method Based on Pseudo Point Cloud Feature Enhancement
    Kong D.-M.
    Li X.-W.
    Yang Q.-X.
    [J]. Jisuanji Xuebao/Chinese Journal of Computers, 2024, 47 (04): : 759 - 775
  • [47] 3D Point Cloud Semantic Segmentation Network Based on Coding Feature Learning
    Tong, Guofeng
    Liu, Yongxu
    Peng, Hao
    Shao, Yuyuan
    [J]. Moshi Shibie yu Rengong Zhineng/Pattern Recognition and Artificial Intelligence, 2023, 36 (04): : 313 - 326
  • [48] A multilevel fusion network for 3D object detection
    Xia, Chunlong
    Wei, Ping
    Wei, Wenwen
    Zheng, Nanning
    [J]. NEUROCOMPUTING, 2021, 437 : 107 - 117
  • [49] PTA-Det: Point Transformer Associating Point Cloud and Image for 3D Object Detection
    Wan, Rui
    Zhao, Tianyun
    Zhao, Wei
    [J]. SENSORS, 2023, 23 (06)
  • [50] A multilevel fusion network for 3D object detection
    Xia, Chunlong
    Wei, Ping
    Wei, Wenwen
    Zheng, Nanning
    [J]. Neurocomputing, 2021, 437 : 107 - 117