Error Awareness by Lower and Upper Bounds in Ensemble Learning

Ensemble learning system could lower down risk of overfitting that often appears in supervised learning for a single learning model. However, overfitting had still been observed in negative correlation learning that trains a set of neural networks simultaneously with correlation-based penalties. In negative correlation learning, each subsystem could see all training data, and focus on those data that has not been learned well by the other subsystems in the ensemble. One cost of learning all data points is that the learned decision boundary could get too closer to some data points. Such decision boundary might not give the better generalization even if it could provide the higher accuracy on the training data. Two constraints are introduced into negative correlation learning for preventing overfitting. One is the lower bound of error rate (LBER). The other is the upper bound of error output (UBEO). These two error bounds would decide whether to learn a certain data point. Experimental results would explore how LBER and UBEO would lead negative correlation learning towards a better decision boundary.