Measuring 3-D understanding on the Web and in the laboratory

We describe the ongoing development of a psychometric test to measure understanding of three-dimensional (3-D) concepts represented in drawings. The test consists of five subtests requiring participants to complete timed choice tasks previously used in experimental investigations of 3-D understanding, and a new subtest based on the idea of true length. We examined performance in terms of accuracy and response time using both laboratory- and Web-based methods of delivery. Reliability coefficients were high for both methods (.90 and .96, respectively, for accuracy, and .87 and .95, respectively, for response time measures), but Web-based participants produced consistently lower overall scores. Issues regarding the deployment of, and recruitment for, complex Web-based experimental tasks, and factors affecting comparisons between Web and laboratory results, such as the presence of an experimenter, and interface and sample differences, are discussed.