A Fusion Deep Learning Model of ResNet and Vision Transformer for 3D CT Images

The outbreak of COVID-19 has had a serious impact on the safety of human life and property. Rapid and effective diagnosis is the key to the prevention and treatment of the virus. In this study, we introduce a new fusion model called "Reswin", which was trained by 3D CT data to diagnose COVID-19. The model combines two mainstream computer vision models, Resnet 3D (a convolutional neural network) and Video Swin Transformer (a vision transformer neural network), which use a soft voting method. We compared our proposed model Reswin with ResNet 3D-50, Swin-T, MViT, R2+1 D-50, SlowFast-50, X3D, and CSN101, which are state-of-the-art deep learning models used for the classification of 3D images. The Reswin model achieved an accuracy of 0.9099, precision of 0.9266, F1 score of 0.9425, AUC of 0.9541, and AUPR of 0.9861 in binary classification, and an accuracy of 0.8655, precision of 0.8580, and F1 score of 0.8620 in triple classification. Reswin provides a new solution for 3D CT image classification tasks and new ideas for the development of deep learning in 3D medical imaging.