Relationship between the quality of service provided through store-and-forward telemedicine consultations and the difficulty of the cases - implications for long-term quality assurance

We examined the difficulty of telemedicine cases and the quality of the resultant consultation in a mature store-and-forward telemedicine network. A random sample of 10 telemedicine cases was selected from those occurring over a 3-month period (5% of the workload) and they were scored by three experienced observers. Inter-observer agreement on the difficulty scores was poor (Fleiss's kappa = 0.18) and it was also poor on the consultation quality scores (Fleiss's kappa = 0.11). Differences between observers were minimized by consensus scoring, and the cases were re-assessed jointly by two observers. Based on the consensus scores, there was a weak negative relation between output quality and case difficulty, i.e., the more difficult cases tended to result in lower quality consultations. However, the effect was non-significant (P = 0.59) and a larger study might be helpful. In the meantime, routine monitoring of telemedicine service quality will continue in the interests of quality assurance. As yet, there is no evidence on which to base a correction for case difficulty.