Multi-Classification of Brain Tumors on Magnetic Resonance Images Using an Ensemble of Pre-Trained Convolutional Neural Networks

Background Automatic classification of brain tumors is an important issue in computer-aided diagnosis (CAD) for medical applications since it can efficiently improve the clinician's diagnostic performance and the current study focused on the CAD system of the brain tumors. Methods Existing studies mainly focused on a single classifier either based on traditional machine-learning algorithms or deep learning algorithms with unsatisfied results. In this study, we proposed an ensemble of pre-trained convolutional neural networks to classify brain tumors into three types from their T1-weighted contrast-enhanced MRI (CE-MRI) images, which are meningioma, glioma, and pituitary tumor. Three pre-trained convolutional neural networks (Inception-v3, Resnet101, Densenet201) with the best classification performance (i.e. accuracy of 96.21%, 97.00%, 96.54%, respectively) on the CE-MRI benchmark dataset were selected as backbones of the ensemble model. The features extracted by backbone networks in the ensemble model were further classified by a support vector machine. Results The ensemble system achieved an average classification accuracy of 98.14% under a five-fold cross-validation process, outperforming any single deep learning model in the ensemble system and other methods in the previous studies. Performance metrics for each brain tumor type, including area under the curve, sensitivity, specificity, precision, and F-score, were calculated to show the ensemble system's performance. Our work addressed a practical issue by evaluating the model with fewer training samples. The classification accuracy was reduced to 97.23%, 96.87%, and 93.96% when 75%, 50%, and 25% training data was used to train the ensemble model, respectively. Conclusion Our ensemble model has a great capacity and achieved the best performance in any single convolutional neural networks for brain tumors classification and is potentially applicable in real clinical practice.