Who Should Be Selected to Perform a Task in Crowdsourced Testing?

被引:23
|
作者
Cui, Qiang [1 ,4 ]
Wang, Junjie [1 ]
Yang, Guowei [2 ]
Xie, Miao [1 ,4 ]
Wang, Qing [1 ,3 ,4 ]
Li, Mingshu [1 ,3 ,4 ]
机构
[1] Chinese Acad Sci, Inst Software, Lab Internet Software Technol, Beijing, Peoples R China
[2] Texas State Univ, Dept Comp Sci, San Marcos, TX USA
[3] Chinese Acad Sci, Inst Software, State Key Lab Comp Sci, Beijing, Peoples R China
[4] Univ Chinese Acad Sci, Beijing, Peoples R China
基金
中国国家自然科学基金; 美国国家科学基金会;
关键词
D O I
10.1109/COMPSAC.2017.265
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Crowdsourced testing is an emerging trend in software testing, which relies on crowd workers to accomplish test tasks. Due to the cost constraint, a test task usually involves a limited number of crowd workers. Furthermore, more workers does not necessarily result in detecting more bugs. Different workers, who may have different testing experience and expertise, may make much differences in the test outcomes. For example, some inappropriate workers may miss true bug, introduce false bugs or report duplicated bugs, which decreases the test quality. In current practice, a test task is usually dispatched in a random manner, and the quality of testing cannot be guaranteed. Therefore, it is important to select an appropriate subset of workers to perform a test task to ensure high bug detection rate. This paper introduces ExReDiv, a novel hybrid approach to select a set of workers for a test task. It consists of three key strategies: the experience strategy selects experienced workers; the relevance strategy selects workers with expertise relevant to the given test task; the diversity strategy selects diverse workers to avoid detecting duplicated bugs. We evaluate ExReDiv based on 42 test tasks from one of the largest crowdsourced testing platforms in China, and the experimental results show its effectiveness.
引用
收藏
页码:75 / 84
页数:10
相关论文
共 50 条