Gaussian-Mixture Neural Networks

Density estimation is crucial to statistical pattern recognition, in both the supervised and unsupervised frameworks. It is still an open problem, due to its intrinsic difficulties and to the many shortcomings of statistical parametric and non-parametric techniques. Artificial neural networks (ANN) have long been applied to the estimation of posterior probabilities for pattern classification (a simple supervised learning task), yet only a few attempts have been made to devise ANN-based density estimation algorithms. The paper proposes a novel algorithm for training an ANN from an unlabeled data sample of patterns randomly drawn from an underlying probability density function (PDF), say p(center dot), such that the ANN learns a robust non-parametric model of p(center dot). The algorithm leverages both the generalization capabilities of ANNs and the generality of the maximum-likelihood estimates of the parameters of Gaussian mixture models. Therefore, the proposed machine is termed Gaussian-mixture Neural Network (GNN). The best selling points of the GNN lie in its simplicity and effectiveness. Preliminary experimental results are reported and analyzed that involve data samples of variable size randomly drawn from PDFs of known form, either unimodal or multimodal. The GNN favorably compares with established statistical and ANN-based estimators, scoring generally higher than its competitors over a range of evaluation metrics. The code used in the experiments is made publicly available online on GitHub.