Systematic review to establish absolute standards for technical performance in surgery

Background: Standard setting allows educators to create benchmarks that distinguish between those who pass and those who fail an assessment. It can also be used to create standards in clinical and simulated procedural skill. The objective of this review was to perform a systematic review of the literature using absolute standard-setting methodology to create benchmarks in technical performance. Methods: A systematic review was conducted by searching MEDLINE, Embase, PsycINFO and the Cochrane Database of Systematic Reviews. Abstracts of retrieved studies were reviewed and those meeting the inclusion criteria were selected for full-text review. The quality of evidence presented in the included studies was assessed using the Medical Education Research Study Quality Instrument (MERSQI), where a score of 14 or more of 18 indicates high-quality evidence. Results: Of 1809 studies identified, 37 used standard-setting methodology for assessment of procedural skill. Of these, 24 used participant-centred and 13 employed item-centred methods. Thirty studies took place in a simulated environment, and seven in a clinical setting. The included studies assessed residents (26 of 37), fellows (6 of 37) and staff physicians (17 of 37). Seventeen articles achieved a MERSQI score of 14 or more of 18, whereas 20 did not meet this mark. Conclusion: Absolute standard-setting methodologies can be used to establish cut-offs for procedural skill assessments.