Variability in the interpretation of grey-scale ultrasound features in assessing thyroid nodules: A systematic review and meta-analysis

Purpose: To investigate the interobserver agreement of different thyroid imaging report and data system (TI-RADS) and ultrasound (US) features. Methods: Two researchers independently searched PubMed, Embase and the Web of Science for relevant studies published between October 1972 and December 2018. Studies investigating interobserver agreement between different radiologists were included. The Guidelines for Reporting Reliability and Agreement Studies (GRRAS) and the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) score tools were applied to assess the quality of the studies. The data for the inter-agreements of TI-RADS categories and ultrasound features were extracted, and combined with STATA 12.0 (StataCorp, College Station, TX) was used. Results: Seven studies including 927 patients were eligible for this meta-analysis. There was a moderate variability in the TI-RADS categories among radiologists (0.54; 95% CI: 0.49-0.58). Regarding the US features, the reliability of composition (0.61; 95% CI: 0.55-0.66) and calcification (0.71; 95% CI: 0.65-0.77) was good, the reliability of echogenicity (0.58; 95% CI: 0.51-0.64), shape (0.53; 95% CI: 0.45-0.62), margin (0.40; 95% CI: 0.32-0.48) and echogenic foci (0.43; 95% CI: 0.32-0.54) was moderate. Subgroup analyses showed that experience/training, the number of observers and the number of patients were the main factors influencing the variability. Conclusions: The interobserver agreement for the TI-RADS categories was moderate. There remains potential for improvement, especially in terms of the echogenicity, shape, margin and echogenic foci, the precision of the description and the targeted training needed.