Subject-Based Transfer Learning in Longitudinal Multiple Sclerosis Lesion Segmentation

Background and Purpose: Accurate and consistent lesion segmentation from magnetic resonance imaging is required for longitudinal multiple sclerosis (MS) data analysis. In this work, we propose two new transfer learning-based pipelines to improve segmentation performance for subjects in longitudinal MS datasets. Method: In general, transfer learning is used to improve deep learning model performance for the unseen dataset by fine-tuning a pretrained model with a limited number of labeled scans from the unseen dataset. The proposed methodologies fine-tune the deep learning model for each subject using the first scan and improve segmentation performance for later scans for the same subject. We also investigated the statistical benefits of the proposed methodology by modeling lesion volume over time between progressors according to confirmed disability progression and nonprogressors for a large in-house dataset (937 MS patients, 3210 scans) using a linear mixed effect (LME) model. Results: The results show statistically significant improvement for the proposed methodology compared with the traditional transfer learning method using Dice (improvement: 2%), sensitivity (6%), and average volumetric difference (16%), as well as visual analysis for public and in-house datasets. The LME result showed that the proposed subject-wise transfer learning method had increased statistical power for the measurement of longitudinal lesion volume. Conclusion: The proposed method improved lesion segmentation performance and can reduce manual effort to correct the automatic segmentations for final data analysis in longitudinal studies.