A Deep Network for Explainable Prediction of Non-imaging Phenotypes Using Anatomical Multi-view Data

Large datasets often contain multiple distinct feature sets, or views, that offer complementary information that can be exploited by multi-view learning methods to improve results. We investigate anatomical-multi-view data, where each brain anatomical structure is described with multiple feature sets. In particular, we focus on sets of white matter microstructure and connectivity features from diffusion MRI, as well as sets of gray matter area and thickness features from structural MRI. We investigate machine learning methodology that applies multi-view approaches to improve the prediction of non-imaging phenotypes, including demographics (age), motor (strength), and cognition (picture vocabulary). We present an explainable multi-view network (EMV-Net) that can use different anatomical views to improve prediction performance. In this network, each individual anatomical view is processed by a view-specific feature extractor and the extracted information from each view is fused using a learnable weight. This is followed by a wavelet-transform-based module to obtain complementary information across views which is then applied to calibrate the view-specific information. Additionally, the calibrator produces an attention-based calibration score to indicate anatomical structures' importance for interpretation. In the experiments, we demonstrate that the proposed EMV-Net significantly outperforms several state-of-the-art methods designed for nonimaging phenotype prediction based on the Human Connectome Project (HCP) Young Adult dataset with 1065 individuals. EMV-Net significantly outperforms compared methods for predicting age, strength, and picture vocabulary. Specifically, our approach specifically decreases the Mean Absolute Error (MAE) for age prediction by at least 0.24 years and improves the correlation coefficient for predicting the other two phenotypes by at least 0.13. Our interpretation results show that for different views, fractional anisotropy of white matter diffusion measures and the surface thickness of gray matter measures are generally more important.