A NOTE ON LEAST-SQUARES LEARNING PROCEDURES AND CLASSIFICATION BY NEURAL NETWORK MODELS

In recent years there has been considerable interest in the capabilities of neural network models applied to tasks such as classification [1]-[4], which typically require some a posteriori judgment of likelihood (for example, of class membership) based upon probabilistic data. This has led to analyses of network learning and function in a statistical context [5]-[8]. We consider neural network models as mathematical classifiers whose inputs comprise random variables generated according to arbitrary stationary class distributions, and address the implication of learning based upon minimization of sum-square classification error over a training set of these observations for which class assignments are absolutely determined. Expectations for network outputs in such case are weighted least-squares approximations to a posteriori probabilities for the classes, which justifies interpretation of network outputs as indicating degree of confidence in class membership. We demonstrate this with a straightforward proof, in which class probability densities are regarded as primitives, and which for simplicity does not rely upon probability theory or statistics. In addition, we cite more detailed results giving conditions for consistency of the estimators (with respect to the weighted least-squares approximations) [9], and discuss some issues relating to the suitability of neural network models and back-propagation training [10] for approximation of conditional probabilities in classification tasks.