Improved Filter-Based Feature Selection Using Correlation and Clustering Techniques

Feature engineering and feature selection are essential techniques to most data science and machine learning applications, in which, respectively, raw data are transformed into features and features are selected to provide the most effective subset of features for the application. Feature selection techniques are particularly useful when dealingwith high-dimensional datasets that contain noisy and redundant data. An optimised feature subset could enhance the performance as well as the interpretability of the model. There are three types of feature selection methods, namely filter, wrapper and embedded techniques. Amongst these methods, the filter method is more efficient than the others as it is computationally less expensive and more generalised. This work presents two improved filter-based feature selection methods based on a correlation coefficient and clustering techniques. The first approach is based on feature correlation where the feature subset consists of features above a similarity threshold to identify a kind of neighbourhood for each feature. The second method uses clustering analysis on the correlation data to identify features that can be used to represent the entire cluster. The obtained feature subsets have been applied as pre-processing step for logistic regression and artificial neural networks. The performance of the proposed methods has been compared against the popular ReliefF feature selection method. The experimental analysis shows that the proposed feature selection methods provide an observable improvement in accuracy by choosing the most effective features.