Equipping Learners to Evaluate Online Health Care Resources: Longitudinal Study of Learning Design Strategies in a Health Care Massive Open Online Course

Background: The digital revolution has led to a boom in the number of available online health care resources. To navigate these resources successfully, digital literacy education is required. Learners who can evaluate the reliability and validity of online health care information are likely to be more effective at avoiding potentially dangerous misinformation. In addition to providing health care education, massive open online courses (MOOCs) are well positioned to play a role in providing digital literacy education in this context. Objective: This study focused on learners enrolled in a MOOC on cancer genomics. The aim of this study was to evaluate the efficacy of a series of digital literacy-related activities within this course. This was an iterative study, with changes made to digital literacy-related activities in 4 of the 8 runs of the course. Methods: This mixed methods study focused on learner engagement with the digital literacy-related activities, including the final course written assignment. Quantitative data including the number of references listed in each written assignment were compared between successive runs. Qualitative data in the form of learner comments on discussion forums for digital literacy-related tasks were evaluated to determine the impact of these educational activities. Results: Using the number of references included for each final course assignment as an indicator of digital literacy skills, the digital literacy-related activities in the final 2 runs were judged to be the most successful. We found a statistically significant increase in the number of references cited by learners in their final written assignments. The average number of references cited in Run 8 was significantly higher (3.5) than in Run 1 (1.8) of the MOOC (P=.001). Learner comments in Runs 7 and 8 showed that a poll in which learners were asked to select which of 4 online resources was reliable was effective in stimulating learner discussion about how to evaluate resource reliability. Conclusions: Similar to many health care MOOCs, the course studied here had a heterogeneous group of learners, including patients (and their families), the public, health care students, and practitioners. Carefully designing a range of digital literacy-related activities that would be beneficial to this heterogenous group of learners enabled learners to become more effective at evaluating and citing appropriate online resources within their written assignments.