Differences in Resident Self-Evaluation and Clinical Competency Committee Evaluation Using ACGME Milestone Versions 1.0 and 2.0

OBJECTIVE: Intentionally self-driven professional development of surgical resident physicians is a hallmark of surgical training and is expected to gain further traction as Entrustable Professional Activities (EPAs) become the new paradigm for surgical education. We aimed to ana-lyze how surgical residents rate themselves as compared to the evaluation of the Clinical Competency Committee using ACGME Milestones Version 1 (M1.0) and Version 2 (M2.0).DESIGN: We asked 22 general surgical trainees for self-evaluation of Milestones (both M1.0 and M2.0) from 2017 semiannually to 2022. ACGME-required Milestone evaluations by the Clinical Competency Committee (CCC) were independently performed after the time window for resident self-evaluation. Neither trainees nor CCC were aware of the other party's evaluations. There were 1552 paired data available for evaluating individual competencies by both trainees and CCC. Paired Wil-coxon signed-rank tests were then performed among the corresponding pairs.SETTING: MercyOne Des Moines Medical Center, Des Moines, IA; Teaching tertiary referral center.PARTICIPANTS: Twenty-two general surgical trainees at this hospital and 28 faculty surgeons participated in this study.RESULTS: The average self-evaluation of surgical resi-dents was lower in the M1.0 cohort compared to the cor-responding CCC evaluation (1.96 +/- 0.72 vs. 2.11 +/- 0.67; p < 0.001). M1.0 self-assessments and CCC-assessments were statistically similar for ICS (p = 0.548) and PROF (p = 0.554) competencies and differed for MK (p < 0.001), PBLI (p < 0.001), PC (p < 0.001), SBP (p = 0.008). On the contrary, the M2.0 cohort demon-strated higher average self-evaluation of surgical resi-dents compared to the corresponding CCC evaluation (2.75 +/- 0.87 vs. 2.12 +/- 0.97; p < 0.001). Significant differences were observed for all 6 ACGME competencies using M2.0 self-assessments and CCC-assessments (all < 0.001). Multivariate regression modeling (p < 0.001, R2 = 0.255) predicted the degree of discordance between self-assessment and CCC-assessed achievement of compe-tencies with a significant effect of gender (baseline male: coef =-0.232, p < 0.001), PGY level (-0.083 per year, < 0.001) and Milestone version (0.831, p < 0.001). A sig-nificant interaction exists for all gender/Milestone combi-nations except for the female trainees with M1.0.CONCLUSIONS: The difference between self-evaluated Milestone achievement and faculty-driven CCC evalua-tion of surgical resident physician performance is more evident in Milestones 2.0 than in Milestones 1.0. Resi-dents self-evaluate higher compared to faculty using Milestones 2.0. This discrepancy is seen among both genders and is more pronounced among male residents overestimating core competencies with M2.0 self-evalua-tion than formal CCC assessment. ( J Surg Ed 80:1378-1384. (c) 2023 Published by Elsevier Inc. on behalf of Association of Program Directors in Surgery.)