A scoring system for elective triage of referrals: Spine Severity Score

BACKGROUND CONTEXT: The Spine Severity Score (SSS) is a 15-point scoring system devised for the purpose of triaging elective surgical spine referrals. From the referral letter and the accompanying radiology report, a total score is calculated based on clinical, pathological, and radiological criteria; a maximum score of 5 can be obtained within each category. A higher total score represents a referral that should be seen more urgently. PURPOSE: To report interrater and intrarater reliability for the SSS and compare the scoring system against the traditional system for triage, that is, the surgeon's clinical experience. STUDY DESIGN: A prospective cross-comparison design was used to evaluate the reliability and convergent validity of the SSS using spine case referrals. METHODS: Four spine surgeons (experts) and three administrative assistants (nonexperts) at the University of Calgary scored 25 referrals. A second iteration of scoring was performed with a minimum time interval elapsed of 6 weeks. Raters were instructed to choose the most concerning (the one with the highest associated score) descriptor in each category that was thought relevant to the individual referral. No further instructions were given on how to interpret the referral letter or the radiology report. The surgeons also scored the referrals using their own preexisting four-point scoring systems. The results were analyzed with independent and dependent t tests, Pearson product moment correlation coefficient, and generalizability and decision analysis. RESULTS: An independent-measures t test (p>.05) revealed no statistical differences between experts and nonexperts (ie, interrater reliability) for both Iterations 1 and 2 on total scores of the SSS and a moderately strong relationship between their ratings across iterations (r=0.79, p<.001). Similarly, a paired-samples t test (p>.05) indicated a nonsignificant mean difference between Total SSS ratings at Time I and Time 2 (ie, intrarater reliability) and a high degree of agreement (r=0.96, p<.001) between the two iterations. These results were confirmed with correlational analyses. Pearson product moment correlation coefficients between the gold standard and the mean score were calculated from expert ratings on Total SSS at Time 1 (r=0.71, p<.001) and at Time 2 (r=0.69, p<.001). CONCLUSIONS: The SSS is a reliable scoring system for triage of elective spine referrals, even among nonexperts. We have been able to demonstrate strong interrater and intrarater reliability for the SSS and moderately strong correlation with the traditional triage system. (C) 2010 Elsevier Inc. All rights reserved.