Label-free differentiation of pancreatic pathologies from normal pancreas utilizing end-to-end three-dimensional multimodal networks on CT

AIMS: To investigate the utilization of an end-to-end multimodal convolutional model in the rapid and accurate diagnosis of pancreatic diseases using abdominal CT images. MATERIALS AND METHODS: In this study, a novel lightweight label-free end-to-end multimodal network (eeMulNet) model was proposed for the rapid and precise diagnosis of abnormal pancreas. The eeMulNet consists of two steps: pancreatic region localization and multimodal CT diagnosis integrating textual and image data. A research dataset comprising 715 CT scans with various types of pancreas diseases and 228 CT scans from a control group was collected. The training set and independent test set for the multimodal classification network were randomly divided in an 8:2 ratio (755 for training and 188 for testing). RESULTS: The eeMulNet model demonstrated outstanding performance on an independent test set of 188 CT scans (Normal: 45, Abnormal: 143), with an area under the curve (AUC) of 1.0, accuracy of 100%, and sensitivity of 100%. The average testing duration per patient was 41.04 seconds, while the classification network took only 0.04 seconds. CONCLUSIONS: The proposed eeMulNet model offers a promising approach for the diagnosis of pancreatic diseases. It can support the identification of suspicious cases during daily radiology work and enhance the accuracy of pancreatic disease diagnosis. The codes and models of eeMulNet are publicly available at Rudeguy1/eeMulNet (github.com).