Heuristic Speciation for Evolving Neural Network Ensemble

Speciation is an important concept in evolutionary computation. It refers to all enhancements of evolutionary algorithms to generate a set of diverse solutions. The concept is studied intensively in the evolutionary design of neural net,work ensembles. The diversity and cooperation of individual networks are among the essential criteria of the design. This paper proposes a speciation framework for ensemble design which integrates a collection of new techniques. Its characteristic features are: (a) the population of networks are speciated as such that the mutual information between the networks' outputs and genotypic representations is preserved. (b) The ensemble is designed incrementally, upon discovery of a species of networks which enhances the ensemble performance. (c) Multiple species are evolved and individual networks are evaluated according to the role of their respective species in the ensemble. This framework provides all implementation of evolutionary algorithm which performs simultaneous single-objective optimizations. The new algorithm is evaluated with a series of classification benchmarks and shows an improvement over other evolutionary training strategies and a statistical algorithm.