A hybrid algorithm for compact neural network ensemble

This paper presents a neural network ensemble creation method where component networks are determined automatically by sequential training. Only previously misclassifted patterns by existing component networks are used for training coming component networks sequentially. This training strategy forces coming component networks to work only on the unsolved portion of the input space. As a result coming component networks could easily maintain diversity with existing component networks in the ensemble. Finally all component networks are trained simultaneously, while maintaining error interaction among them through a penalty function. This new method has been tested extensively on several benchmark problems of machine learning and neural networks. Experimental result shows that it can produce compact ensemble structure that exhibits good generalization ability.