Camera Based AI Models Used with LiDAR Data for Improvement of Detected Object Parameters

There is an increasing demand for the implementation of autonomous missions that require precise control of special purpose vehicles based on data provided by perception systems. These unmanned ground vehicles (UGVs) operate in unknown environment where accurate location of detected objects is crucial for defining trajectories. Vision systems are major in military vehicles, providing situational awareness necessary for teleoperations. The data from cameras are commonly used by autonomy modules with implemented artificial intelligence (AI) to recognize and classify objects effectively. Unfortunately vision systems face challenges in providing obstacles precise distance estimation due to inherent limitations of the sensors, such as the inability to accurately measure depth information. To overcome this limitation and improve the perception capabilities, autonomous vehicles employ additional active sensors like LiDARs or radars providing reliable distance measurements and other relevant spatial data. The authors proposed simplified approach to data fusion using available and already trained neural network models for object detection and classification as well as correlation with spatial data in the form of flat view from laser scanners to provide precise information about the detected objects. The article discusses different approaches to data fusion in autonomous vehicles' perception system. The possibility of using flat view to extract additional information about localized object features are described. The paper addresses challenges in correlating detected objects in 2D camera images with spatial data from active sensors. The proposed approach can improve autonomous navigation in the operational environment.