Combining feature selection with feature weighting for k-NN classifier

The k-nearest neighbor (k-NN) classification is a simple and effective classification approach. However, it suffers from over-sensitivity problem due to irrelevant and noisy features. In this paper, we propose an algorithm to improve the effectiveness of k-NN by combining these two approaches. Specifically, we select all relevant features firstly, and then assign a weight to each one. Experimental results show that our algorithm achieves the highest accuracy or near to the highest accuracy on all test datasets. It also achieves higher generalization accuracy compared with the well-known algorithms IB1-4 and C4.5.