Generalized Oja's rule for linear discriminant analysis with Fisher criterion

On-line learning rules for both Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) with Fisher criterion are analyzed under the same framework, and a generalized Oja's rule for both is derived. For the LDA problem, the relationship between the Fisher criterion and the criterion of minimizing Mean Square Error (MSE) is discussed. The experiments show that the convergence speed of the generalized Oja's rule as an adaptive method for Fisher Criterion is much faster than that of gradient descent method for MSE criterion.