Scalable evaluation methods for autonomous vehicles

Effective intelligent driving test and evaluation methods can improve the development and deployment processof autonomous vehicles (AVs). However, due to the extreme complexity and high dimensionality of drivingbehavior, how to objectively and efficiently evaluate the multi-dimensional performance of AVs in simulationand real-world environments is a long-standing problem. This paper proposes an objective multi-dimensionalcomprehensive evaluation (OMDCE) method that divides the intelligent driving test into four modules: testscenarios, scenario complexity model, simulation test platform, and automated evaluation system. The scenariocomplexity model is proposed to bridge the gap between the test scenarios and the automated evaluationsystem, thus enabling adaptive scaling of the evaluation scale for test scenarios with different difficultylevels. Besides, based on the existing four evaluation metrics for ego performance, altruism performanceevaluation is first proposed to comprehensively portray AVs' intelligence degree. The OMDCE method wasevaluated by testing two different intelligent driving algorithms and conducting real vehicle testing. Theexperimental results demonstrated that the OMDCE method can effectively evaluate AVs in various scenariosand quantitatively measure their multi-dimensional performance. The proposed method reduces the subjectivityof manual evaluation and speeds up the evaluation. Moreover, it improves the universality and scalability ofthe evaluation metrics system