Inter-Reader Variability Using PI-RADS v2 Versus PI-RADS v2.1: Most New Disagreement Stems from Scores 1 and 2

Purpose To analyze possible differences in the inter-reader variability between PI-RADS version 2 (v2) and version 2.1 (v2.1) for the classification of prostate lesions using multiparametric MRI (mpMRI) of the prostate. Methods In this retrospective and randomized study, 239 annotated and histopathologically correlated prostate lesions (104 positive and 135 negative for prostate cancer) were rated twice by three experienced uroradiologists using PI-RADS v2 and v2.1 with an interval of at least two months between readings. Results were tabulated across readers and reading timepoints and inter-reader variability was determined using Fleiss kappa (kappa). Thereafter, an additional analysis of the data was performed in which PI-RADS scores 1 and 2 were combined, as they have the same clinical consequences. Results PI-PI-RADS v2.1 showed better inter-reader agreement in the peripheral zone (PZ), but poorer inter-reader agreement in the transition zone (TZ) (PZ: kappa = 0.63 vs. kappa = 0.58; TZ: kappa = 0.47 vs. kappa = 0.57). When PI-RADS scores 1 and 2 were combined, the use of PI-RADS v2.1 resulted in almost perfect inter-reader agreement in the PZ and substantial agreement in the TZ (PZ: kappa = 0.81; TZ: kappa = 0.80). Conclusion PI-RADS v2.1 improves inter-reader agreement in the PZ. New differences in inter-reader agreement were mainly the result of the assignment of PI-RADS v2.1 scores 1 and 2 to lesions in the TZ. Combining scores 1 and 2 improved inter-reader agreement both in the TZ and in the PZ, indicating that refined definitions may be warranted for these PI-RADS scores.