Aligning classroom assessment with engineering practice: A design-based research study of a two-stage exam with authentic assessment

Background Authentic assessment and two-stage exams have recently received attention; however, they are rarely used together. We reimagine assessment by integrating an authentic, computer-based assessment into the structure of a two-stage exam in a large engineering class. Purpose We seek to identify ways that such assessment extends classroom testing to better align with engineering practice by examining the ways teams negotiate uncertainty to make engineering decisions. We also identify differing students' reactions to increased uncertainty during tests. Design/Method Using the methodical framework of design-based research, we analyze performance and reflection data for 117 student teams through two design iterations to explore four design and theoretical conjectures. Results Teams chose multiple solution paths to this authentic task, an aspect that aligns with the characteristics of engineering practice that we seek to assess. In addition, the technology tool allows the evaluation of procedural accuracy for many of the teams' chosen paths. The teams' decision-making performances correlate; however, decision-making and traditional assessments do not correlate, suggesting they measure different competencies. The computer-based second stage provides a holistic assessment that shifts the messages that students implicitly receive about valued practices in the classroom. However, not all students took up the authentic group assessment in desired ways. Conclusions Technology-based two-stage exams with authentic assessment show promise to shift testing practices in large engineering classes to include decision-making. Such assessments better align with engineering practices that are valued in the profession, but more work is needed to develop systems for widespread implementation.