Time complexity analysis of neural networks on message passing multicomputer systems

One of the main disadvantages of Artificial Neural Networks (ANN) is the long time periods required for training which is due to the large numbers of interconnections between the neurons required for many real-world problems. ANNs are amenable for parallel implementations due to their inherent parallelism. In this paper, we propose some time-cost complexity models for the training parallelism and algorithmic parallelism approaches to implementing backpropagation on a multilayer perceptron ANN architecture. The models are compared with empirical results obtained from a network of T-800 transputers and are found to be accurate enough to be used to predict the performance of the algorithms for different problem sizes and for different message passing multicomputer systems. From these models we are then able to theoretically determine the size of the multicomputer required for a certain problem size so as to obtain the best performance.