The Design and Experimental Evaluation of a Scaffolded Software Environment to Improve Engineering Students' Disciplinary Problem-Solving Skills

BACKGROUND Introductory gateway engineering courses are notorious for their high attrition rates. Deficiencies in students' problem-solving processes may contribute to their failure in these courses. In an empirical study of student problem solving, we observed that students struggle because of misconceptions regarding the basic syntax and semantics of disciplinary diagrams and corresponding mathematical equations. PURPOSE (HYPOTHESIS) We hypothesize that a scaffolded software environment that provides dynamically-generated feedback on the syntactic and semantic correctness of students' evolving disciplinary diagrams and mathematical equations can improve engineering students' problem-solving abilities. DESIGN/METHOD We iteratively developed ChemProV, a software environment to assist chemical engineering students in solving material balance problems. To evaluate ChemProV's effectiveness, we performed two between-subjects experimental studies. The first study compared a preliminary version of the ChemProV to pen-and-paper. The second study compared a redesigned version of ChemProV with dynamic feedback to the same version of ChemProV without dynamic feedback. RESULTS While it did not uncover any significant differences, the first study provided insight into how to improve ChemProV's dynamic feedback mechanism. The second study found that the "feedback" version of ChemProV promoted a statistically-significant advantage in problem-solving accuracy, significantly more time-on-task, and a transfer-of-training to an unscaffolded problem-solving situation. CONCLUSIONS A scaffolded software environment like ChemProV can serve as a valuable aid in helping students learn engineering problem-solving skills. Its software design approach can be used as a model for designing educationally-effective software environments for other engineering disciplines.