An Optimal Design Method for Artificial Neural Networks by Using the Design of Experiments

This paper presents a method to optimally design artificial neural networks with many design parameters using the Design of Experiment (DOE), whose features are efficient experiments using an orthogonal array and quantitative analysis by analysis of variance. Neural networks can approximate arbitrary nonlinear functions. The accuracy of a trained neural network at a certain number of learning cycles depends on both weights and biases and its structure and learning rate. Design methods such as trial-and-error, brute-force approaches, network construction, and pruning, cannot deal with many design parameters such as the number of elements in a layer and a learning rate. Our design method realizes efficient optimization using DOE, and obtains confidence of optimal design through statistical analysis even though trained neural networks very due to randomness in initial weights. We apply our design method three-layer and five-layer feedforward neural networks in a preliminary study and show that approximation accuracy of multilayer neural networks is increased by picking up many more parameters.