Towards Systematic Live Experimentation in Software-Intensive Systems of Systems

As the size, variation, and sophistication of software-intensive systems-of-systems grows, so does the uncertainty inherent to their design and development. To deal with this issue, we propose a framework for systematic experimentation based on declarative specification connected with system architecture. The focus is on how to specify experiments that allow systematic exploration of the space of alternative configurations at runtime. Since such experiments should be launched on live systems, extra care needs to be taken in preventing damages when experimenting with the systems. Therefore, we also focus on how to quantify the direct and the indirect cost associated with each experiment execution (which needs to be included in a cost-benefit analysis for system adaptation) and on how to gradually roll out an experiment via a number of different stages. We use the development of a route planner system as an example to motivate and exemplify our approach.