Orthogonal Relief Algorithm for Feature Selection

The Relief is a popular feature selection algorithm. However, it is ineffective in removing redundant features due to its feature evaluation mechanism that all discriminative features are assigned with high relevance scores, regardless of the correlations in between. In the present study, we develop an orthogonal Relief algorithm (O-Relief) to tackle the redundant feature problem. The basic idea of the O-Relief algorithm is to introduce an orthogonal transform to decompose the correlation between features so that the relevance of a feature could be evaluated individually as it is done in the original Relief algorithm. Experiment results on four world problems show that the orthogonal Relief algorithm provides features leading to better classification results than the original Relief algorithm.