Comparative case studies of open source software peer review practices

Context: The power of open source software peer review lies in the involvement of virtual communities, especially users who typically do not have a formal role in the development process. As communities grow to a certain extent, how to organize and support the peer review process becomes increasingly challenging. A universal solution is likely to fail for communities with varying characteristics. Objective: This paper investigates differences of peer review practices across different open source software communities, especially the ones engage distinct types of users, in order to offer contextualized guidance for developing open source software projects. Method: Comparative case studies were conducted in two well-established large open source communities, Mozilla and Python, which engage extremely different types of users. Bug reports from their bug tracking systems were examined primarily, complemented by secondary sources such as meeting notes, blog posts, messages from mailing lists, and online documentations. Results: The two communities differ in the key activities of peer review processes, including different characteristics with respect to bug reporting, design decision making, to patch development and review. Their variances also involve the designs of supporting technology. The results highlight the emerging role of triagers, who bridge the core and peripheral contributors and facilitate the peer review process. The two communities demonstrate alternative designs of open source software peer review and their trade-offs were discussed. Conclusion: It is concluded that contextualized designs of social and technological solutions to open source software peer review practices are important. The two cases can serve as learning resources for open source software projects, or other types of large software projects in general, to cope with challenges of leveraging enormous contributions and coordinating core developers. It is also important to improve support for triagers, who have not received much research effort yet. (C) 2015 Elsevier B.V. All rights reserved.