A field experiment on trialsourcing and the effect of contract types on outsourced software development

Context: To ensure the success of software projects, it is essential to select skilled developers and to use suitable work contracts. Objective: This study tests two hypotheses: (i) the use of work-sample testing (trialsourcing) improves the selection of skilled software developers; and (ii) the use of contracts based on hourly payment leads to better software project outcomes than fixed-price contracts. Method: Fifty-seven software freelancers with relevant experience and good evaluation scores from previous clients were invited to complete a two-hour long trialsourcing task to qualify for a software development project. Thirty-six developers completed the trialsourcing task with acceptable performance, and, based on a stratified random allocation process, were asked to give a proposal based on an hourly payment or a fixed-price contract. Eight hourly payment-based and eight fixed-priced proposals were accepted. The process and product characteristics of the completion of these 16 projects were collected and analysed. Results and Conclusion: While the use of trialsourcing may have prevented the selection of developers with insufficient skills, the performance on the trialsourcing task of the selected developers did, to a large extent, not predict their performance on the projects. The use of hourly payments seems to have led to lower costs than fixed-price contracts, but not to improved processes or products. We plan to follow up these results with research on how to design more skill-predictive trialsourcing tasks, and when and why different project contexts give different contract consequences.