Use of an Analytical Grading Rubric for Self-Assessment: A Pilot Study for a Periodontal Oral Competency Examination in Predoctoral Dental Education

While educators agree that using self-assessment in education is valuable, a major challenge is the poor agreement often found between faculty assessment and student self-assessment. The aim of this study was to determine if use of a pre-defined grading rubric would improve reliability between faculty and dental student assessment on a periodontal oral competency examination. Faculty members used the grading rubric to assess students' performance on the exam. Immediately after taking the exam, students used the same rubric to self-assess their performance on it. Data were collected from all third- and/or fourth-year students in four classes at one U.S. dental school from 2011 to 2014. Since two of the four classes took the exam in both the third and fourth years, those data were compared to determine if those students' self-assessment skills improved over time. Statistical analyses were performed to determine agreement between the two faculty graders and between the students' and faculty assessments on each criterion in the rubric and the overall grade. Data from the upper and lower performing quartiles of students were sub-analyzed. The results showed that faculty reliability for the overall grades was high (K=0.829) and less so for individual criteria, while student-faculty reliability was weak to moderate for both overall grades (Spearman's rho=0.312) and individual criteria. Students in the upper quartile self-evaluated themselves more harshly than the faculty (p<0.0001), while the lower quartile students overestimated their performance (p=0.0445) compared to faculty evaluation. No significant improvement was found in assessment over time in the students who took the exam in the third and fourth years. This study found only limited support for the hypothesis that a grading rubric used by both faculty and students would increase correspondence between faculty and student assessment and points to a need to reexamine the rubric and instructional strategies to help students improve their ability to self-assess their work.