Assessing Computational Thinking: Development and Validation of the Algorithmic Thinking Test for Adults

This study describes the development and validation process of a computational thinking (CT) test for adults. The team designed a set of items and explored a subset of those through two qualitative pilots. Then, in order to provide validity evidence based on the test content, a team of 11 subject-matter experts coded the initial pool of items using two different systems of categories based on CT components and contents. Next, the items were piloted on a sample of 289 participants, 137 experts in CT, and 152 novices. After a series of confirmatory factor analyses, a unidimensional model that represents algorithmic thinking was adopted. Further analyzing the psychometric quality of the 27 items, 20 of them with excellent reliability indices were selected for the test. Thus, this study provides a tool to evaluate adults' CT: the Algorithmic Thinking Test for Adults (ATTA), which was developed according to psychometric standards. This article also reflects on the nature of CT as a construct.