A Pairwise Algorithm for Training Multilayer Perceptrons with the Normalized Risk-Averting Error Criterion

Proper use of the normalized risk-averting error (NRAE) criterion has been shown to avoid nonglobal local minima effectively in the mean squared error (MSE) criterion. For training on large datasets, a pairwise algorithm for the NRAE criterion similar to the widely-used least mean square algorithm for the MSE criterion is proposed. The gradual deconvexification method employing this pairwise algorithm is tested on examples with built-in nonglobal local minima that are difficult to avoid and on recognition of handwritten numerals with the MNIST dataset. Numerical experiments show that the pairwise algorithm for the NRAE criterion is computationally more economical than the corresponding batch algorithm and delivers multilayer perceptrons with better performances than training methods based on the MSE criterion.