Assessing Knowledge and Application of the Design Process

Authentic, client-based projects form the foundation of a one-semester freshman design course at Rice University. The course is an elective course available for all freshman students in the School of Engineering. First-year students learn the engineering design process and use it to solve meaningful problems drawn from local hospitals, industry, local community partners, Rice University, and international partners. The objectives for the course are to (a) have students learn and practice the engineering design process early in their engineering education, and (b) increase undergraduate retention in engineering at Rice University by 10 percentage points. These two objectives align with important themes and goals published elsewhere. For this first-year design course, three specific learning outcomes have been defined: students design a product that meets user-defined needs and realistic constraints; students communicate effectively through written reports and oral/visual presentations; and students work effectively on multidisciplinary teams. Steps in the design process form the core of the course lecture material. Most class periods are split between an interactive lecture about a step in the design process and team meetings to complete the design process, including prototype construction. Assessment of students' knowledge of the design process was measured by asking students to critique the strengths and weaknesses of a Gantt chart. The Gantt chart laid out a 14-week design process in which a team was to develop and build an examination bed for a clinic built by Engineers Without Borders in Nicaragua. Using this recently refined assessment tool, students' knowledge and application of the design process was evaluated at the beginning and end of the semester. Written responses were coded by three trained raters on eight aspects: 1) needs assessment/establishing design criteria; 2) design context review; 3) idea generation; 4) analysis and decision-making; 5) building and testing; 6) overall layout of a design process and iteration; 7) time allotments; and 8) documentation. Data presented in the paper includes 72 student responses from fall 2012 and spring 2013. Analysis shows statistical significance between pre- and post-test results for seven of the eight topics evaluated. The refined assessment method described here is an improvement over a previously deployed assessment tool.