The nature of qualitative comments in evaluating professionalism

INTRODUCTION We analysed written comments on clerkship evaluation forms to determine if they furthered our understanding of students' professional behaviour. METHODS We transcribed all written comments related to professional behaviour for 1 medical school class in 4 major clerkships, then categorised the comments. Comments were coded into categories by two raters. They were also coded as positive, negative or equivocal. The 3 types of comments were each correlated with a Likert-type score for professionalism for each student in these same clerkships. RESULTS Most comments covered initiative, working well with others, patient skills and working hard. Of the 1845 comments, 1721 were positive, 106 were negative and 18 were equivocal, neither wholly positive nor wholly negative. The total number of positive comments correlated with the students' numerical professionalism score (r = 0.29, P < 0.001); negative comments correlated negatively with the numerical professionalism score (r = -0.44, P < 0.001). Equivocal comments were significantly negative (r = -0.25, P = 0.002). Qualitative analysis revealed that equivocal comments were focused on maturity, initiative and teamwork. CONCLUSIONS Written comments contain unique indicators of students' professional behaviour and are largely positive in nature; negative comments stand out by their rarity and intensity. There is a set of negative comments masked by evaluators in language that appears neutral or even partly positive that statistically resemble negative comments and should be regarded as such by students who receive this type of comments. Evaluators should be encouraged and trained to express their qualitative views of students.