Random effects models for assessing diagnostic accuracy of traditional Chinese doctors in absence of a gold standard

Two common problems in assessing the accuracy of traditional Chinese medicine (TCM) doctors in detecting a particular symptom are the unknown true symptom status and the ordinal-scale of the symptom status. Wang et al. (Biostatistics 2011; DOI: 10.1093/biostatistics/kxq075) proposed a nonparametric maximum likelihood method for estimating the accuracy of different TCM doctors without a gold standard when the true symptom status is measured on an ordinal-scale. A key assumption of their work is that the diagnosis results are independent conditional on the gold standard. This assumption can be violated in many practical situations. In this paper, we propose a random effects modeling approach that extends their method to incorporate dependence structure among different tests or doctors. The proposed method is illustrated on a real data set from TCM, which contains the diagnostic results from five doctors for the same patients regarding symptoms related to Chills disease. The same data set was analyzed by Wang et al. under the conditional independence assumption. In addition, we also discuss an ad hoc test for the model fitting and a likelihood ratio test on the random effects. Copyright (c) 2011 John Wiley & Sons, Ltd.