Comparison of two standard-setting methods for advanced cardiac life support training

Background This study used the Angoff and Hofstee standard-setting methods to derive minimum passing scores for six advanced cardiac life support (ACLS) procedures. Method An expert panel provided item-based (Angoff) and group-based (Hofstee) judgments about six ACLS performance checklists on two occasions separated by ten weeks. Interrater reliabilities and test-retest reliability (stability) of the judgments were calculated. Derived ACLS passing standards are compared to historical ACLS performance data from two groups of ACLS-trained internal medicine residents. Results Both the Angoff and Hofstee standard-setting methods produced reliable and stable data. Hofstee minimum passing scores (MPSs) were uniformly more stringent than Angoff MPSs. Interpretation of historical ACLS performance data from medical residents shows the MPSs derived in this study would yield higher-than-expected failure rates. Conclusion Systematic standard setting for ACLS procedures is a necessary step toward the creation of mastery learning educational programs.