Deep Metric Learning for K Nearest Neighbor Classification

K Nearest Neighbor (kNN) has gained popularity in machine learning due to its simplicity and good performance. However, kNN faces two problems with classification tasks. The first is that an appropriate distance measurement is required to compute distances between test sample and training samples. The other is the highly computational complexity due to the requirement of searching the nearest neighbors in the whole training data. In order to mitigate these two problems, we propose a novel method named KCNN to enhance the performance of kNN. KCNN uses convolutional neural networks (CNN) to learn a suitable distance metric as well as prototype reduction to learn a reduced set of prototypes which can represent the original set. It has several superiorities compared with related methods. The combination of CNN and kNN empowers it to extract discriminative hierarchical features with which kNN can easily classify. KCNN learns spatial information on an image instead of considering it as a vector to learn distance metric. Moreover, KCNN simultaneously learns a reduced set of prototypes, which help improve classification efficiency and avoid noisy samples of the massive training set. The proposed method has a better robustness and convergence than CNN, especially when projecting input data into a rather low-dimension space.