Fault Types of Adaptive and Context-Aware Systems and Their Relationship with Fault-based Testing Approaches

Context: Testing adaptive systems (ASs) and context-aware systems (CASs) is usually more challenging than testing conventional systems. This is due to their characteristics, such as the high number of possible configurations and runtime adaptations. Therefore, it is important to know the types of faults that may be incurred by these systems so that such faults can be either avoided during implementation, or detected through tests (e.g. in a fault-based testing approach such as mutation). Objective: Characterising and grouping types of faults for ASs and CASs that have been previously described in the literature, but are disperse in various studies. Moreover, identifying which and how the fault types are addressed by the current research on fault-based testing. Method: we used a Systematic Literature Review that addressed testing for ASs and CASs, whose one of the goals was a characterisation of fault types for ASs and CASs. By dealing with such a goal, we analysed 11 primary studies that address fault types. We also used code snippets to illustrate how the fault types may occur. Finally, we analysed the faults that are addressed in other 7 studies that investigated fault-based testing for ASs and CASs. Results: We present lists of specific fault types (26 in total) and fault type categories (6 in total) for AS and CASs, and discuss the relationship between these fault types with current fault-based testing approaches. Conclusion: Our results are original when compared to the state-of-the-art, given that we proposed the first characterisation of fault types for ASs and CASs. Beyond this, we point out room for improvement of fault-based testing of these types of systems, particularly regarding the types of faults addressed by the current research.