The Pass/Fail Effect: A Longitudinal Study of United States Medical Licensing Examination (USMLE) Step 1 Performance Over a Decade

Objectives This study aimed to analyze the impact of the United States Medical Licensing Examination (USMLE) Step 1 transition to a pass/fail scoring system in 2022 on the performance of first-time test takers in three distinct groups: Doctor of Osteopathy (DO) and Doctor of Medicine (MD) examinees from US/Canadian schools and examinees from non-US/Canadian schools. The analysis spans a decade-long period from 2012 to 2022, offering insights into the implications of this pivotal change in medical education. Methods We analyzed the performance of first-time USMLE Step 1 examinees from US/Canadian MD and DO programs and non-US/Canadian schools from 2012 to 2022, including the transition year to a pass/fail scoring system. Data were obtained from USMLE performance data reports and organized into annual contingency tables. Descriptive statistics and comparative analysis were used to identify trends and differences in performance across the groups. Data visualization techniques were employed to illustrate these findings, and the results were contextualized within the broader changes in medical education. Results In 2021, first-time takers from US/Canadian MD and DO Degree programs had pass rates of 96% and 94%, respectively, while non-US/Canadian schools had a pass rate of 82%. However, in 2022, these rates dropped to 93%, 89%, and 74%, respectively. The most significant relative decline was observed among non-US/ Canadian Schools' first-time takers, with an 8% decrease. Repeaters consistently had lower pass rates across all groups. Conclusion The study reveals a notable decline in pass rates following the transition to pass/fail scoring, although this is based on just one year of data. This underscores the importance of students not rushing into the exam and dedicating sufficient time for preparation. The potential impact of this research could be transformative for medical education, but more years of data post-transition will be needed to confirm these initial findings. These findings serve as a reminder that the change in scoring does not diminish the rigor of the exam, prompting students to approach their studies with diligence and patience and potentially paving the way for systemic improvements in medical education and healthcare delivery worldwide.