Considerations for the use of commercial video games in controlled experiments

While commercial, off-the-shelf video games have been used often to observe cognitive and social phenomena, few studies have taken advantage of such games as research tools for conducting controlled experiments. Providing evidence that unmodified, commercial video games can be used to conduct gaming evaluations with high levels of both experimental control and ecological validity, we designed and conducted an experiment that utilized Mario Kart Wii to evaluate the effects of natural interaction on player performance. Based on our experience from that experiment, we present several concerns that require attention when using commercial video games as research tools. Providing examples of design decisions and outcomes from our experiment, we identify some of the benefits, drawbacks, and challenges of using such tools. (C) 2011 International Federation for Information Processing Published by Elsevier B.V. All rights reserved.