Explainable Artificial Intelligence Requirements for Safe, Intelligent Robots

While requirements for robot performance to perform a task are generally well understood, the requirements around the explanatory capabilities of these systems are often at best an afterthought. This results in a dangerous situation where neither users nor experts can predict situations where the robot will or will not work, nor understand what causes failures and unexpected behaviour. In this paper, we discuss and survey the field of Explainable Artificial Intelligence, as it relates to the generation of requirements for the development of safe, intelligent robots. We then present a categorisation of explanatory capabilities and requirements that aims to help users and developers alike to ensure an appropriate match between the types of explanations that a given application requires, and the capabilities of various underlying AI techniques.