Fisher plus Kernel criterion for discriminant analysis

We simultaneously approach two tasks of nonlinear discriminant analysis and kernel selection problem by proposing a unified criterion, Fisher+Kernel Criterion. In addition, an efficient procedure is derived to optimize this new criterion in an iterative manner. More specifically, original input vector is first transformed into a higher dimensional feature matrix through a battery of nonlinear mappings involved in different kernels. Then, based on the feature matrices, FKC is presented within two coupled projection spaces: one projection space is used to search for the optimal combinations of kernels; while the other encodes the optimal nonlinear discriminating projection directions. Our proposed method is a unified framework for both kernel selection and nonlinear discriminant analysis. Besides, the algorithm potentially alleviates overfitting problem existing in traditional KDA and has no singularity problems in most cases. The effectiveness of our proposed algorithm is validated by extensive face recognition experiments on several datasets.