The feasibility and effectiveness of a blended-learning course for detecting and avoiding bias in medical data: a pilot study

Background Conflicts of interest (COIs), including those arising from interactions with pharmaceutical companies, may lead to bias in medical data. Although medical students are now requesting more education on COIs and bias, they are still not adequately taught during medical school, and few published courses on this topic exist. The objective of our study was therefore to evaluate the feasibility and effectiveness of a blended-learning course for detecting and avoiding bias in medical data, with a special focus on COIs. Methods We developed a blended learning course on bias detection, COIs, and risk communication. It was piloted in the Fall Semester of 2019/2020 using a pre/post-test design. The primary outcome was a gain in bias detection skills, tested by a novel key feature test. Secondary outcomes were (i) skepticism (tested using an attitude questionnaire), (ii) the intention to manage COIs in a professional way so as to avoid bias (tested using a situational judgment test) and (iii) the course evaluation by the students. Results Seventeen students participated in the study. The key feature test showed a significant improvement in bias detection skills at post-testing, with a difference in means of 3.1 points (95%-CI: 1.7-4.4, p-value: < 0.001; highest possible score: 16 points). The mean score after the course was 6.21 (SD: 2.62). The attitude questionnaire and situational judgment test also showed an improvement in skepticism and intentions to manage COIs, respectively. Students evaluated the course as having been worthwhile (Median: 5, IQR: 0.75, Likert-Scale 1-6, 6 = fully applicable). Conclusions The blended learning format of the course was feasible and effective. The results suggest a relevant learning gain; however, the low mean score on the key feature test after the course reflects the difficulty of the subject matter. Although a single course has the potential to induce significant short-term improvements in bias detection skills, the complexity of this important subject necessitates its longitudinal integration into medical curricula. This concept should include specific courses such as that presented here as well as an integration of the topic into clinical courses to improve context-related understanding of COIs and medical data bias.