Enhancing heart failure diagnosis through multi-modal data integration and deep learning

In the realm of medical data processing, the surge in electronic health records has opened avenues for addressing clinical challenges. Although machine and deep learning methods have gained traction, they often overlook the potential of multimodal data. Thus, multimodal fusion emerges as a prominent field in artificial intelligence research, capitalizing on the synergy between diverse data types to enhance classification models. This study introduces an innovative technique tailored for heart failure classification, harnessing the power of multimodal features. The proposed approach utilizes three distinct feature types: electrocardiogram, chest X-ray, and structured text data. These are integrated to form a comprehensive multimodal fusion model. This study demonstrates the superior performance of the proposed model compared to single-modality counterparts, even in the presence of noise, through a rigorous experiment involving 440 cases. It pioneers the integration of multimodal information using deep learning techniques for heart failure assessment, offering novel insights and a practical approach for accurate detection and treatment.