Scalable prediction of non-functional properties in software product lines: Footprint and memory consumption

Context: A software product line is a family of related software products, typically created from a set of common assets. Users select features to derive a product that fulfills their needs. Users often expect a product to have specific non-functional properties, such as a small footprint or a bounded response time. Because a product line may have an exponential number of products with respect to its features, it is usually not feasible to generate and measure non-functional properties for each possible product. Objective: Our overall goal is to derive optimal products with respect to non-functional requirements by showing customers which features must be selected. Method: We propose an approach to predict a product's non-functional properties based on the product's feature selection. We aggregate the influence of each selected feature on a non-functional property to predict a product's properties. We generate and measure a small set of products and, by comparing measurements, we approximate each feature's influence on the non-functional property in question. As a research method, we conducted controlled experiments and evaluated prediction accuracy for the non-functional properties footprint and main-memory consumption. But, in principle, our approach is applicable for all quantifiable non-functional properties. Results: With nine software product lines, we demonstrate that our approach predicts the footprint with an average accuracy of 94%, and an accuracy of over 99% on average if feature interactions are known. In a further series of experiments, we predicted main memory consumption of six customizable programs and achieved an accuracy of 89% on average. Conclusion: Our experiments suggest that, with only few measurements, it is possible to accurately predict non-functional properties of products of a product line. Furthermore, we show how already little domain knowledge can improve predictions and discuss trade-offs between accuracy and required number of measurements. With this technique, we provide a basis for many reasoning and product-derivation approaches. (C) 2012 Elsevier B.V. All rights reserved.