A Model for the Measurement of the Runtime Testability of Component-based Systems

Runtime testing is emerging as the solution for the integration and validation of software systems where traditional development-time integration testing cannot be performed, such as Systems of Systems or Service Oriented Architectures. However performing tests during deployment or in-service time introduces interference problems, such as undesired side-effects in the state of the system or the outside world. This paper presents a qualitative model of runtime testability that complements Binder's classical restability model, and a generic measurement framework for quantitatively assessing the degree of runtime testability of a system based on the ratio of what can be tested at runtime vs. what would have been tested during development time. A measurement is devised for the concrete case of architecture-based test coverage, by using a graph model of the systems architecture. Concretely, two testability studies are performed for two component based systems, showing how to measure the runtime testability of a system.