Evaluation of End-To-End Learning for Autonomous Driving: The Good, the Bad and the Ugly

End-to-End learning for autonomous vehicles has received much attention recently. Departing from traditional complex systems, it promises to address the steering, throttle, and braking predictions with a single neural network. However, it requires significant amounts of diverse training data, high-end cameras and expensive computation units. In this paper we describe an end-to-end learning platform for autonomous driving, built with emphasis on minimizing requirements, both in terms of software and hardware. We accomplish autonomous navigation on a closed track with a single low-cost camera and a typical laptop. We describe each step, from data pre-processing to actuation, evaluate trade-offs, and motivate decisions. Our analysis concludes that standard performance metrics are incomplete; thus we expand our analysis to heuristic behavioral evaluation by visualizing the backpropagation process, broadening our understanding of the deep neural network.