Optimizing Artificial Neural Network for Functions Approximation Using Particle Swarm Optimization

Artificial neural networks (ANN) are commonly used in function approximation as well as classification problems. This paper shows a configurable architecture of a simple feed forward neural network trained by particle swarm optimization (PSO) algorithm. PSO and ANN have several hyperparameters that have impact on the results of approximation. ANN parameters are the number of layers, number of neurons in each layer, and neuron activation functions. The hyperparameters of the PSO are the population size, the number of informants per particle, and the acceleration coefficients. Herein, this work comes to spot the light on how the PSO hyperparameters affect the ability of the algorithm to optimize ANNs weights in the function approximation task. This was examined and tested by generating multiple experiments on different types of input functions such as: cubic, linear, XOR problem. The results of the proposed method show the superiority of PSO compared to backpropagation in terms of MSE.