Reducing Class Overlapping in Supervised Dimension Reduction

Dimension reduction is to find a low-dimensional subspace to project high-dimensional data on, such that the discriminative property of the original higher-dimensional data is preserved. In supervised dimension reduction, class labels are integrated into the lower-dimensional representation, to produce better results on classification tasks. The supervised dimension reduction (SDR) framework by [17] is one of the state-of-the-art methods that takes into account not only the class labels but also the neighborhood graphs of the data, and have some advantages in preserving the within-class local structure and widening the between-class margin. However, the reduced-dimensional representation produced by the SDR framework suffers from the class overlapping problem - in which, data points lie closer to a different class rather than the class they belong to. The class overlapping problem can hurt the quality on the classification task. In this paper, we propose a new method to reduce the overlap for the SDR framework in [17]. The experimental results show that our method reduces the size of the overlapping set by an order of magnitude. As a result, our method outperforms the pre-existing framework on the classification task significantly. Moreover, visualization plots show that the reduced-dimensional representation learned by our method is more scattered for within class data and more separated for between-class data, as compared to the pre-existing SDR framework.