Task-selection bias: A case for user-defined tasks

Usability evaluations typically occur throughout the life cycle of a product. A number of decisions and practical biases concerning the tasks selected for Usability evaluations can influence the results. A pervasive bias is to select only tasks that are possible to perform with the product under evaluation, introducing a subtle bias for the participants. One way to avoid this problem is to employ user-defined tasks (UDTs) in usability evaluations. In addition, having participants define tasks to perform in a product evaluation allows a more accurate assessment of product usability. This is because UDTs based on users' requirements and expectations should be relatively independent of the functional capabilities of a product. However, there are a number of methodological and practical issues that result from the introduction of UDTs in a usability evaluation. The best approach is to design hybrid evaluations using both UDTs and product-supported tasks.