Multi-Modal Multi-Stream UNET Model for Liver Segmentation

Computer segmentation of abdominal organs using CT and MRI images can benefit diagnosis, treatment, and workload management. In recent years, UNETs have been widely used in medical image segmentation for their precise accuracy. Most of the UNETs current solutions rely on the use of single data modality. Recently, it has been shown that learning from more than one modality at a time can significantly enhance the segmentation accuracy, however most of available multi-modal datasets are not large enough for training complex architectures. In this paper, we worked on a small dataset and proposed a multi-modal dual-stream UNET architecture that learns from unpaired MRI and CT image modalities to improve the segmentation accuracy on each individual one. We tested the practicality of the proposed architecture on Task 1 of the CHAOS segmentation challenge. Results showed that multi-modal/multi-stream learning improved accuracy over single modality learning and that using UNET in the dual stream was superior than using a standard FCN. A "Dice" score of 96.78 was achieved on CT images. To the best of our knowledge, this is one of the highest reported scores yet.