Chernoff-Based Multi-class Pairwise Linear Dimensionality Reduction

Linear dimensionality reduction techniques have been studied very well for the two-class problem, while the corresponding issues encountered when dealing with multiple classes are far from trivial. In this paper, we show that dealing with multiple classes, it is not expedient to treat it as a multi-class problem, but it is better to treat it as an ensemble of Chernoff-based two-class reductions onto different subspaces. The solution is achieved by resorting to either Voting, Weighting, or a Decision Tree combination scheme. The ensemble methods were tested on benchmark datasets demonstrating that the proposed method is not only efficient, but also yields an accuracy comparable to that obtained by the optimal Bayes classifier.