Improving detection of web service antipatterns using crowdsourcing

Web services design may suffer lousy design choices, i.e., antipatterns. These antipatterns often lead to software that is difficult to maintain and evolve. So far, manual and automated methods have been proposed for identifying these antipatterns. But these methods either require a lot of time by professionals or have the problem of uncertainty. This paper presents a solution based on crowdsourcing. This solution can improve the performance of other methods by using crowd wisdom and teamwork. The proposed crowdsourcing solution is introduced in four phases, including task design, task assignment and submission, task validation, and task aggregation. First, the services are placed in a repository to be distributed among different users. The antipatterns detection of an instance of a service is assigned to two divisions of agents and ordinary users. Then the feedbacks are submitted, and the reliability of this feedbacks is determined. The proposed reliability mechanism checks the bias, lack of user expertise, or spam by examining the consistency with the majority view and the trend of user feedback and filters out unreliable comments. Finally, feedbacks are aggregated. Two methods of user study and simulation are considered for performance evaluation. The user study is conducted to gain the performance and comparison of the proposed approach. This study indicates that crowdsourcing does not have a bias toward any specific technologies. The Mann-Whitney test also reveals a significant difference with other approaches with an average precision and recall scores of 91% and 94%. The simulation is also performed to study the long-term behavior of users. The results show that the proposed approach could push out biased feedbacks.