Comparing neural networks and data analysis

Multi Layer Perceptrons (MLPs) trained through the Gradient Back propagation algorithm have recently been introduced in a large class of applications, where Pattern Recognition techniques were used before. We present here a theoretical framework which allows to compare MLPs with Data Analysis techniques such as Principal Component Analysis and Discriminant Analysis. Our results establish the equivalence between a linear MLP, with one hidden layer and either PCA or DA. They can be easily extended to MLPs with more hidden layers. We briefly sketch the elements of our theory (for Discriminant Analysis). We illustrate these theoretical results with simulations which demonstrate that MLPs with linear elements do as well as DA or PCA, but that MLPs with non linear elements and more than p hidden units can largely outperform PCA and DA on auto-association and hetero-association tasks.