Systematic literature review on the impacts of agile release engineering practices

Context: Agile release engineering (ARE) practices are designed to deliver software faster and cheaper to end users; hence, claims of such impacts should be validated by rigorous and relevant empirical studies. Objective: The study objective was to analyze both direct and indirect impacts of ARE practices as well as to determine how they have been empirically studied. Method: The study applied the systematic literature review research method. ARE practices were identified in empirical studies by searching articles for "rapid release," "continuous integration," "continuous delivery," and "continuous deployment." We systematically analyzed 619 articles and selected 71 primary studies for deeper investigation. The impacts of ARE practices were analyzed from three viewpoints: impacts associated with adoption of the practice, prevalence of the practice, and success of software development. Results: The results indicated that ARE practices can create shorter lead times and better communication Within and between development teams. However, challenges and drawbacks were also found in change management, software quality assurance, and stakeholder acceptance. The analysis revealed that 33 out of 71 primary studies were casual experience reports that had neither an explicit research method nor a data collection approach specified, and 23 out of 38 empirical studies applied qualitative methods, such as interviews, among practitioners. Additionally, 12 studies applied quantitative methods, such as mining of software repositories. Only three empirical studies combined these research approaches. Conclusion: ARE practices can contribute to improved efficiency of the development process. Moreover, release stakeholders can develop a better understanding of the software project's status. Future empirical studies should consider the comprehensive reporting of the context and how the practice is implemented instead of merely referring to usage of the practice. In addition, different stakeholder points of view, such as customer perceptions regarding ARE practices, still clearly require further research. (C) 2017 Elsevier B.V. All rights reserved.