Examiners are most lenient at the start of a two-day OSCE

Background: OSCEs can be both reliable and valid but are subject to sources of error. Examiners become more hawkish as their experience grows, and recent research suggests that in clinical contexts, examiners are influenced by the ability of recently observed candidates. In OSCEs, where examiners test many candidates over a short space of time, this may introduce bias that does not reflect a candidate's true ability. Aims: Test whether examiners marked more or less stringently as time elapsed in a summative OSCE, and evaluate the practical impact of this bias. Methods: We measured changes in examiner stringency in a 13 station OSCE sat by 278 third year MBChB students over the course of two days. Results: Examiners were most lenient at the start of the OSCE in the clinical section (beta = -0.14, p = 0.018) but not in the online section where student answers were machine marked (beta = -0.003, p = 0.965). Conclusions: The change in marks was likely caused by increased examiner stringency over time derived from a combination of growing experience and exposure to an increasing number of successful candidates. The need for better training and for reviewing standards during the OSCE is discussed.