Initial experience with a multi-station objective structured teaching skills evaluation

Purpose. To explore the applicability of using an objective structured evaluation to assess community preceptors' teaching performances. Method. The authors developed and, in 1996, administered an objective structured teaching evaluation (OSTE) at the University of California, Irvine, College of Medicine. They compared assessments of teaching skills made by faculty observers and standardized students, and examined instrument reliability, realism, and utility. Results. Reliabilities of the OSTE varied with individual stations but, in several instances, approached acceptable standards. Faculty observers' and standardized students' evaluations of preceptors' performances were similar. Participants' feedback indicated that the greatest OSTE-related benefit was derived by the faculty observers. Conclusions. An OSTE that reflects realistic teaching situations can be successfully developed. Objective structured evaluation can be successfully applied to assessing faculty teaching performance. However, it may be no more discriminating than are student evaluations. An OSTE's utility may be greatest in guiding faculty development initiatives.