An empirical study on the class imbalance handling techniques for different diseases

Machine learning and deep learning-based techniques are now widely used to identify and diagnose various diseases. However, getting a sufficient data for these machine learning models is difficult, and usually collected data is unbalanced i.e. less number of instances with disease class and a large number of classes without the disease. These imbalanced data cause poor performance of the classifier in the detection of minority or disease classes. To address this class imbalance problem for medical data we have applied 17 different class imbalance handling techniques on four publically available datasets with Random forest as a base classifier. The comprehensive review covering the time frame of 1990 to 2023 is also done on “Class imbalance handling techniques” to gather insights using VOSviewer software. Performances of different class imbalance handling techniques are statistically evaluated and impact of different disease datasets on the prediction performance is also statistically assessed. Two novel techniques Genetic Algorithm-Cost sensitive-Deep neural network(GA-CS-DNN) and Class imbalance handling technique-Genetic Algorithm-Deep neural Network(CIH-GA-DNN)are proposed for handling class imbalance problems. Performances of proposed techniques are compared with other state of art class imbalance handling techniques and obtained results showed that OnesidedSelection outperformed all other techniques. A statistical test further demonstrated that OnesidedSelection performs differently than SMOTENN. Significant statistical differences in illness prediction can be seen between the kidney and diabetes, prostate and kidney, and kidney and heart datasets when compared pair-wise. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.