Cognitively Inspired Task Design to Improve User Performance on Crowdsourcing Platforms

Recent research in human computation has focused on improving the quality of work done by crowd workers on crowd-sourcing platforms. Multiple approaches have been adopted like filtering crowd workers through qualification tasks, and aggregating responses from multiple crowd workers to obtain consensus. We investigate here how improving the presentation of the task itself by using cognitively inspired features affects the performance of crowd workers. We illustrate this with a case-study for the task of extracting text from scanned images. We generated six task-presentation designs by modifying two parameters - visual saliency of the target fields and working memory requirements - and conducted experiments on Amazon Mechanical Turk (AMT) and with an eyetracker in the lab setting. Our results identify which task-design parameters (e.g. highlighting target fields) result in improved performance, and which ones do not (e.g. reducing the number of distractors). In conclusion, we claim that the use of cognitively inspired features for task design is a powerful technique for maximizing the performance of crowd workers.