The Reliability of Visual Ratings of Velopharyngeal Physiology for Speech

Background: Analyze intrarater and interrater reliability for evaluating endoscopic images of velopharyngeal (VP) physiology. Method: Speakers produced 9 speech stimuli representing 4 stimulus types: sustained phonemes, repetitions of "puh," single words, and short phrases. The 37-speaker participants included 16 patients with VP dysfunction and 21 control participants. Five raters independently rated the video images for degree of VP opening, location of opening, and pattern of closure. Outcome measures included intrarater and interrater measures of reliability and the effects of raters and stimulus type on ratings. Results: Intrarater reliability was acceptable, and ratings were logically consistent. Fixed effects regression coefficients for the patient and the control groups showed that raters were a significant source of variability for degree of opening and pattern of closing. Stimulus type was not a significant source of variation for any metric for the controls, but stimulus type was a significant determinant for degree of opening for patients. The degree of opening was larger for sustained phonemes than for the other speech stimuli. Ratings for degree of opening were most similar for repeated "puh." Conclusions: Interrater reliability needs to be improved so that the assessment procedure produces more consistent findings among clinicians, thus strengthening our evidence base for this procedure. Interrater additional research is needed to understand how the stimulus affects ratings of VP physiology, to identify stimuli that yield the most useful clinical information, and to understand how training affects the ratings of VP physiology.