Research on Multi-tasking Smart Cars Based on Autonomous Driving Systems

Using fewer sensors to achieve more autonomous driving tasks has become a challenge. Some of the existing research implements a single neural network model and achieves the goal of input information diversification and multi-task autonomous driving functionality by adding external devices. However, it adds the burden of designing complex modules. Existing methods with only one sensor (camera) can only achieve two tasks of road tracking and obstacle avoidance. In contrast to existing research, this paper proposed a novel Autonomous Driving System (ADS) that is capable of performing five tasks simultaneously with a single camera: road tracking (Task 1), turn sign recognition (Task 2), lane changing and obstacle avoidance when encountering other cars (Task 3), stopping when encountering obstacles (Task 4), and traffic light recognition (Task 5). The ADS fuses the End-to-end model (Efficientnet_b0-SA-RE) and the Safety-assisted model (Efficientnet_b0-CA), which can be switched between to perform autonomous driving tasks depending on the different road conditions. Additionally, a new Robustness Error (RE) loss function and a new Smoothing Activation (SA) function are presented to optimize both models. The ADS also contains our proposed novel Custom Conversion Scheme (CCS), which integrates TensorRT and half-precision technology (FP16) for accelerating the model, and successfully deploys the dual model on the embedded device Jetson Nano. Furthermore, the untrained environment Track 1–4 is designed to test smart cars. The experimental results demonstrate that our method can complete five tasks simultaneously and achieve a F1 score of 85% in the most challenging Track 4, which is 32% higher than existing methods. © 2023, The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd.