Multi-modal deep learning for predicting progression of Alzheimer's disease using bi-linear shake fusion

Alzheimer's Disease (AD) which causes declination of cognitive function is one of the most severe social issues in the world. It has already been known that AD cannot be cured and treatment can only delay its progression. Therefore, it is very important to detect AD in early stage and prevent it to be worse. Furthermore, sooner the progression is detected, better the prognosis will be. In this research, we developed a novel multi-modal deep learning method to predict conversion from Mild Cognitive Impairment (MCI), which is the stage between cognitively normal older people and AD. In our method, the multi-modal input data are defined as structural Magnetic Resonance Imaging (MRI) images and clinical data including several cognitive scores, APOE genotype, gender and age obtained from Alzheimer's Disease Neuroimaging Initiative cohort (ADNI). Our criteria of selecting these input data are that they are mostly obtained by non-invasive examination. The proposed method integrates features obtained from MRI images and clinical data effectively by using bi-linear fusion. Bi-linear fusion computes the products of all elements between image and clinical features, where the correlation between them are included. That led to a big improvement of prediction accuracy in the experiment. The prediction model using bi-linear fusion achieved to predict conversion in one year with 0.86 accuracy, comparing with 0.76 accuracy using linear fusion. The proposed method is useful for screening examination for AD or deciding a stratification approach within clinical trials since it achieved a high accuracy while the input data is relatively easy to be obtained.