Study of Usage Patterns and Learning Gains in a Web-based Interactive Static Course

BACKGROUND Courseware for engineering education can feature many discrete interactive learning elements, and typically student usage is not compelled. To take advantage of such courseware, self-regulation of learning may be necessary. Evaluation of courseware should consider actual usage, learning gains, and indications of learning self-regulation. PURPOSE (HYPOTHESIS) The research question focuses on how students' interactions with the courseware affect their learning gains. The hypothesis tested is that learning gains from online courseware increase with usage, and particularly with usage that suggests learning self-regulation. DESIGN/METHOD Students in a lecture-based statics course were assigned to study previously developed courseware as part of homework assignments. Learning gains were deduced from pre- and post- paper and pencil diagnostic quizzes, and from the first class exam. Credit was based on quiz scores, rather than courseware usage. Usage of interactive elements of the courseware was inferred from log files of students' interactions with the courseware, and patterns suggesting learning self-regulation were identified. RESULTS High, statistically significant learning gains were found. Substantial usage was evident, with core learning activities initiated by, on average, three-quarters of students. Learning gains and performance on the relevant class exam appeared to be more closely correlated with usage that indicated self-regulation of learning rather than with total usage of the courseware. CONCLUSIONS Methods of assessing courseware should go beyond courseware features, learning gains, and student self-reports of effectiveness to include monitoring of actual usage and analyses relating usage to learning. Self-regulation of learning is likely to be critical to successful usage of courseware, and courseware should be designed to encourage it.