Robust Testing for Cyber-Physical Systems using Reinforcement Learning

In this paper, we propose a testing framework for cyber-physical systems (CPS) that operate in uncertain environments. Testing such CPS applications requires carefully defining the environment to include all possible realistic operating scenarios that the CPS may encounter. Simultaneously, the process of testing hopes to identify operating scenarios in which the system-under-test (SUT) violates its specifications. We present a novel approach of testing based on the use of deep reinforcement learning for robust testing of a given SUT. In a robust testing framework, the test generation tool can provide meaningful and challenging tests even when there are small changes to the SUT. Such a method can be quite valuable in incremental design methods where small changes to the design does not necessitate expensive test generation from scratch. We demonstrate the efficacy of our method on three example systems in autonomous driving implemented within a photo-realistic autonomous driving simulator.