Obtaining accurate neural network ensembles

The main contribution of this paper is to suggest a novel technique for automatic ensemble design, maximizing accuracy. The technique proposed first trains a large number of classifiers (here neural networks) and then uses genetic algorithms to select the members of the final ensemble. The proposed method, when evaluated on 22 publicly available data sets, results in ensembles obtaining very high accuracy, most often outperforming "typical standard ensembles". The study also shows that ensembles created using the straightforward approach of always selecting a fixed number (here five or ten) of top ranked networks results in very accurate ensembles. The conclusion is that the main reason for the increased accuracy is the possibility to select classifiers from a large pool. We argue that this is an important result, since it provides a data miner with an automatic tool for finding high-accuracy models, thus reducing the need for early decisions regarding techniques and model design.