How Explainable Is Your System? Towards a Quality Model for Explainability

[Context and motivation] Explainability is a software quality aspect that is gaining relevance in the field of requirements engineering. The complexity of modern software systems is steadily growing. Thus, understanding how these systems function becomes increasingly difficult. At the same time, stakeholders rely on these systems in an expanding number of crucial areas, such as medicine and finance. [Question/problem] While a lot of research focuses on how to make AI algorithms explainable, there is a lack of fundamental research on explainability in requirements engineering. For instance, there has been little research on the elicitation and verification of explainability requirements. [Principal ideas/results] Quality models provide means and measures to specify and evaluate quality requirements. As a solid foundation for our quality model, we first conducted a literature review. Based on the results, we then designed a user-centered quality model for explainability. We identified ten different aspects of explainability and offer criteria and metrics to measure them. [Contribution] Our quality model provides metrics that enable software engineers to check whether specified explainability requirements have been met. By identifying different aspects of explainability, we offer a view from different angles that consider different goals of explanations. Thus, we provide a foundation that will improve the management and verification of explainability requirements.