Heuristic multi-modal integration framework for liver tumor detection from multi-modal non-enhanced MRIs

The detection of liver tumors from non-enhanced Magnetic Resonance Imaging (MRI) has become crucial for current diagnosis and treatment due to the avoidance of contrast-agent injection and associated health risks. A recent study, based on Deep Reinforcement Learning (DRL), validated the feasibility of detecting liver tumors from non-enhanced MRIs for the first time. However, this study only employed single-modal MRIs, where malignant tumors are often invisible, leading to the detection of only benign tumors. This paper proposes the Heuristic Multi-modal Integration (HMI) framework to detect both benign and malignant tumors from multi-modal non-enhanced MRIs. The HMI utilizes individual DRL modules on each modality to extract specific features and then integrates these modules into a collective DRL module, utilizing the comprehensive information from multiple modalities to detect the desired tumors. Compared to existing liver tumor detection methods, the HMI is the first study without contrast agents in both training and testing, expanding the reach of non-enhanced detection technology. Furthermore, the HMI successfully employs DRL in a multi -modal environment, offering a solution to the difficulties of DRL convergence with complex inputs and high computational requirements. The experimental results show that the HMI outperforms current state-of-the-art methods, making it an accurate and contrast-agent-free alternative for liver tumor detection in clinical settings.