Development and validation of an evaluation instrument for medical students in tutorials

Purpose. To develop and explore the validity of the Tutotest, the first standardized instrument designed to assess the skills and attitudes of medical students working in tutorials in a problem-based learning curriculum. Method. The Tutotest, consisting of 44 items rated on a four-point Likert-type scale, was developed from well-defined theoretical frameworks after consultation with students and teachers at the University de Sherbrooke Faculty of Medicine. In 1988 - 89, 40 tutors were asked to use the Tutotest to evaluate a total of 100 students at the end of each of the four units of the first-year curriculum. Data were analyzed to examine the homogeneity and reliability of the instrument, explore its factorial structure, and document its correlation with students' grades. Results. A total of 28 tutors (70%) conducted Tutotest evaluations; of a possible 400 evaluations, 270 (67.5%) were returned with complete data. Exploratory factor analysis revealed that the Tutotest was structured around four factors that accounted for 82% of the variance: effectiveness in the group (23 items), communication and leadership skills (13 items), scientific curiosity (four items), and respect for colleagues (four items). The Tutotest had a Cronbach's alpha coefficient of .98 and a split-half coefficient of .98. The intraclass correlation coefficient was estimated to be .46 and would reach up to .81 after averaging five Tutotest evaluations. Tutotest scores showed goad correlation (r = .64) with the official tutor's global evaluation and a moderate correlation (r = .39) with students' written examination results. Conclusion. The Tutotest was found to be a standardized reliable and valid instrument that can significantly improve the evaluation of students' skills and attitudes during tutorials. Although lengthier than the usual tutor evaluation form, the Tutotest is a good compromise between an unreliable short global scale and a more comprehensive but impractical instrument.