DepthNet: Real-Time LiDAR Point Cloud Depth Completion for Autonomous Vehicles

被引:24
|
作者
Bai, Lin [1 ]
Zhao, Yiming [1 ]
Elhousni, Mahdi [1 ]
Huang, Xinming [1 ]
机构
[1] Worcester Polytech Inst, Dept Elect & Comp Engn, Worcester, MA 01609 USA
来源
IEEE ACCESS | 2020年 / 8卷
关键词
Laser radar; Three-dimensional displays; Convolution; Autonomous vehicles; Real-time systems; Neural networks; Cameras; LiDAR; point cloud; depth completion; convolutional neural network; FPGA;
D O I
10.1109/ACCESS.2020.3045681
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Autonomous vehicles rely heavily on sensors such as camera and LiDAR, which provide real-time information about their surroundings for the tasks of perception, planning and control. Typically, a LiDAR can only provide sparse point cloud owing to a limited number of scanning lines. By employing depth completion, a dense depth map can be generated by assigning each camera pixel a corresponding depth value. However, the existing depth completion convolutional neural networks are very complex that requires high-end GPUs for processing, and thus they are not applicable to real-time autonomous driving. In this article, a light-weight network is proposed for the task of LiDAR point cloud depth completion. With an astonishing 96.2% reduction in the number of parameters, it still achieves comparable performance (9.3% better in MAE but 3.9% worse in RMSE) to the state-of-the-art network. For real-time embedded platforms, depthwise separable technique is applied to both convolution and deconvolution operations and the number of parameters decreases further by a factor of 7.3, with only a small percentage increase in error performance. Moreover, a system-on-chip architecture for depth completion is developed on a PYNQ-based FPGA platform that achieves real-time processing for HDL-64E LiDAR at the speed 11.1 frame per second.
引用
收藏
页码:227825 / 227833
页数:9
相关论文
共 50 条
  • [1] Real-time depth completion based on LiDAR-stereo for autonomous driving
    Wei, Ming
    Zhu, Ming
    Zhang, Yaoyuan
    Wang, Jiarong
    Sun, Jiaqi
    [J]. FRONTIERS IN NEUROROBOTICS, 2023, 17
  • [2] Real-Time LiDAR Point Cloud Semantic Segmentation for Autonomous Driving
    Xie, Xing
    Bai, Lin
    Huang, Xinming
    [J]. ELECTRONICS, 2022, 11 (01)
  • [3] AN APPROACH TO REAL-TIME COLLISION AVOIDANCE FOR AUTONOMOUS VEHICLES USING LIDAR POINT CLOUDS
    Sandu, C.
    Susnea, I.
    [J]. JOURNAL OF APPLIED ENGINEERING SCIENCES, 2022, 12 (01) : 129 - 134
  • [4] Real-Time LiDAR Point-Cloud Moving Object Segmentation for Autonomous Driving
    Xie, Xing
    Wei, Haowen
    Yang, Yongjie
    [J]. SENSORS, 2023, 23 (01)
  • [5] Real-time 3D LiDAR Flow for Autonomous Vehicles
    Baur, Stefan A.
    Moosmann, Frank
    Wirges, Sascha
    Rist, Christoph B.
    [J]. 2019 30TH IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV19), 2019, : 1288 - 1295
  • [6] PointNet on FPGA for Real-Time LiDAR Point Cloud Processing
    Bai, Lin
    Lyu, Yecheng
    Xu, Xin
    Huang, Xinming
    [J]. 2020 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), 2020,
  • [7] Real-Time Fast Channel Clustering for LiDAR Point Cloud
    Zhang, Xiao
    Huang, Xinming
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, 2022, 69 (10) : 4103 - 4107
  • [8] Quantitative Comparison of LiDAR Point Cloud Segmentation for Autonomous Vehicles
    Anand, Bhaskar
    Barsaiyan, Vivek
    Senapati, Mrinal
    Rajalakshmi, P.
    [J]. 2021 IEEE 94TH VEHICULAR TECHNOLOGY CONFERENCE (VTC2021-FALL), 2021,
  • [9] Real-Time Spatio-Temporal LiDAR Point Cloud Compression
    Feng, Yu
    Liu, Shaoshan
    Zhu, Yuhao
    [J]. 2020 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2020, : 10766 - 10773
  • [10] Real-Time Point Cloud Clustering Algorithm Based on Roadside LiDAR
    Wu, Jianqing
    Zhuang, Xucai
    Tian, Yuan
    Cheng, Zhiheng
    Liu, Shijie
    [J]. IEEE SENSORS JOURNAL, 2024, 24 (07) : 10608 - 10619