Topology Optimization for Artificial Neural Networks using Differential Evolution

Backpropagation (BP) training algorithm is the main algorithm for training feedforward artificial neural networks (ANNs). BP is based on gradient descent, thus it converges to a local optimum in the region of the initial solution. Meanwhile, the evolutionary algorithms (EAs) always look for global optimum, however their ability of local search is not as good as the BP algorithm. This paper presents a hybrid system that uses differential evolution with global and local neighborhoods (DEGL), which is a variant of differential evolution (DE), to search for a suitable architecture and a near-optimal set of initial connection weights, and then performs the Levenberg-Marquadt training algorithm, which is a more robust variation of BP, to perform local search from these initial weights. Finally, it is performed a comparison of the performance of the hybrid system DEGL+ANN with the hybrid system DE+ANN and the raw RNA, for classification problems using machine learning benchmarks.