Multimodal Early Fusion Strategy Based on Deep Learning Methods for Cervical Cancer Identification

It is essential to enhance the accuracy of automatic cervical cancer diagnosis by combining multiple forms of information obtained from a patient's primary examination. However, existing multimodal systems are not very effective in detecting correlations between different types of data, leading to low sensitivity but high specificity. This study introduces a deep learning system for automatic diagnosis of cervical cancer by incorporating multiple sources of data. First, a convolutional neural network (CNN) to transform the image database to a vector that can be combined with non-image datasets is used. Subsequently, an investigation of jointly the nonlinear connections between all image and non-image data in a deep neural network is performed. Proposed deep learning-based method creates a unified system that takes advantage of both image and non-image data. It achieves an impressive 89.32% sensitivity at 91.6% specificity when diagnosing cervical intraepithelial neoplasia on a wide-ranging dataset. This result is far superior to any single-source system or prior multimodal approaches.