Efficient Occupancy Grid Mapping and Camera-LiDAR Fusion for Conditional Imitation Learning Driving

Deep neural networks trained end-to-end on demonstrations of human driving have learned to follow roads, avoid obstacles, and take specific turns in intersections to reach a destination. Such conditional imitation learning approach is demonstrated to drive efficiently when deployed on the same training environments, but performance dramatically decreases when deployed to new environments and is not consistent against varying weathers. In this work, a proposed model aims to cope with such two challenges by fusing laser scanner input with the camera. Additionally, a new efficient method of Occupancy Grid Mapping is introduced and used to rectify the model output to further improve the performance. On CARLA simulator urban driving benchmark, the proposed system improves autonomous driving success rate and average distance traveled towards destination on all driving tasks and environments combinations, while it's trained on automatically recorded traces. Autonomous driving success rate generalization improves by 57% and weather consistency improved by around four times.