Longitudinal Detection of Diabetic Retinopathy Early Severity Grade Changes Using Deep Learning

Longitudinal medical image analysis is crucial for identifying the unobvious emergence and evolution of early lesions, towards earlier and better patient-specific pathology management. However, traditional computer-aided diagnosis (CAD) systems for diabetic retinopathy (DR) rarely make use of longitudinal information to improve DR analysis. In this work, we present a deep information fusion framework that exploits two consecutive longitudinal studies for the assessment of early DR severity changes. In particular, three fusion schemes are investigated: (1) early fusion of inputs, (2) intermediate fusion of feature vectors incorporating Spatial Transformer Networks (STN) and (3) late fusion of feature vectors. Exhaustive experiments compared with respect to no-fusion baselines validate that incorporating prior DR studies can improve the referable DR severity classification performance through the late fusion scheme whose AUC reaches 0.9296. Advantages and limitations of the different fusion methods are discussed in depth. We also propose different pre-training strategies which are employed to bring considerable performance gains for DR severity grade change detection purposes.