How Design Science Research Helps Improve Learning Efficiency in Online Conversations

In this design science research paper, we report on our constructing and evaluating an attention-guidance system that we integrated into a computer-supported collaborative learning system. Drawing on social constructivist literature, our proposed design focuses on attracting, retaining, and, if necessary, reacquiring users' attention on task-relevant information in online collaborative literature processing. The investigation involved an experiment across two sections of students in a human-computer interaction course. Results show that the new design allowed users to consistently reflect and evaluate the content of a text as they capitalized on one another's reasoning to resolve misconceptions. Moreover, we found that the new system increased users' perceptions of learning. However, the difference in knowledge gain scores was marginally significant and represented a medium effect size. Interestingly, we found that the attention-guidance system supported more efficient learning. Finally, we discovered that task-oriented reading of text, revisions of incomplete or incorrect ideas, and perceptions of learning mediated the relationship between software system and learning efficiency. We discuss the theoretical and practical implications.