A hybrid approach for lung cancer diagnosis using optimized random forest classification and K-means visualization algorithm

Lung cancer detection has become one of the most challenging oncology problems. It is an arduous task for radiologists to detect nodules based on the naked eye vision. The main goal of this paper is to present a well-defined approach for malignant nodule detection from computed tomography scans and a visualization tool to show how the extracted features are responsible for the malignant cluster. Inspired by hyperparameter optimization and visualization technique, we uniquely deployed a hybrid approach based on an optimized random forest classifier and a K-means visualization tool that tried to best tune the model's hyperparameters to provide the optimal results and visualize the malignant and non-malignant clusters, respectively. Out of the four experiments performed for the hyperparameter optimization, the best model classified malignant and non-malignant cases effectively and achieved a 10-Fold cross-validation accuracy of 92.14% on the LIDC-IDRI dataset. Moreover, the least inertia score and the highest silhouette score obtained by the best visualization configuration were 16.21 and 0.815, respectively. The proposed hybrid approach appeared to be apt for lung cancer diagnosis. The integration of the visualization approach provided the ability to localize the malignant cluster and hence drew inference out of it.