The role of the interview in residency selection: A mixed-methods study

Context Residency programmes invest considerable time and resources in candidate interviews as a result of their perceived ability to reveal important social traits. However, studies examining the ability of interviews to predict resident performance have shown mixed findings, and the role of the interview in candidate evaluation remains unclear. This mixed-methods study, conducted in an anaesthesiology residency programme at a large academic medical centre, examined how interviews contributed to candidate assessment and whether the addition of behavioural questions to interviews altered their role in the evaluation process. Methods During the 2018-2019 residency selection season in the Department of Anesthesiology and Critical Care at the University of Pennsylvania, independent ratings for each interviewee were collected from faculty interviewers. Consensus ratings subsequently established by committee were also collected. Committee meetings were audiorecorded and transcribed for qualitative analysis. Behavioural questions were integrated into half of interview days. Ratings of candidates interviewed on behavioural question days were compared statistically with those of candidates interviewed on non-behavioural question days. Results Qualitative analysis showed that interviewers heavily emphasised candidates' application files in evaluating the interviews. Interviewers focused on candidates' academic records and favoured candidates whose interview behaviours were consistent with their applications and whose applications demonstrated similarities to interviewers' traits. The addition of behavioural questions demonstrated little ability to alter these dynamics. Quantitatively, there were no significant differences in candidate rating outcomes between behavioural and non-behavioural interviewing days, whereas a higher medical school rating and higher score on the United States Medical Licensing Examination Step 1 were associated with a more favourable consensus rating. Conclusions Residency candidates' application files predisposed interviewers' experience and evaluation of interviews, preventing the interviews from providing discrete assessments of interpersonal qualities, even when behavioural questions were included. In the continued effort to perform well-rounded assessments of residency candidates, further research and reflection on the role of interviewing in evaluation are necessary.