Role of Simulated Lidar Data for Training 3D Deep Learning Models: An Exhaustive Analysis

The use of 3D Deep Learning (DL) models for LiDAR data segmentation has attracted much interest in recent years. However, the generation of labeled point cloud data, which is a prerequisite for training DL models, is a highly resource-intensive exercise. Simulated LiDAR data, which are already labeled, provide a cost-effective alternative, but their efficacy and usefulness must be evaluated. This paper examines the role of simulated LiDAR point clouds in training DL models. A high-fidelity 3D terrain model representing the real environment is developed, and the in-house physics-based simulator "Limulator" is used to generate labeled point clouds through various realizations. The paper outlines a few major hypotheses to assess the usefulness of simulated data in training DL models. The hypotheses are designed to assess the role of simulated data alone or in combination with real data or by strategic boosting of minor classes in simulated data. Several experiments are carried out to test these hypotheses. An experiment involves training a DL model, PointCNN in this case, using various combinations of simulated and real LiDAR data and measuring its performance to segment the test data. Results show that training using simulated data alone can produce an overall accuracy (OA) of 89% and the weighted-averaged F1 score of 88.81%. It is further observed that training using a combination of simulated and real data can achieve accuracies comparable to when only a large quantity of real data is employed. Strategic boosting of minor classes in simulated data improves the accuracies of minor classes by up to 23% compared to only real data. Training a DL model using simulated data, due to the ease in its generation and positive impact on segmentation accuracy, can be highly beneficial in the use of DL for LiDAR data. The use of simulated data for training has the potential to minimize the resource-intensive exercise of developing labeled real data.