Acceleration of Neural Network Learning by GPGPU

Recently, graphic boards have come to have higher performance than CPUs as a result of the development of 3DCG and movie processing, and are now widely used due to progress in computer entertainment. Implementation of general-purpose computing on GPU (GPGPU) has become easier as a result of the integrated development environment CUDA distributed by NVIDIA. A GPU has dozens or hundreds of arithmetic circuits, whose allocations are controlled by CUDA. In prior research, the implementation of a neural network using GPGPU has been studied; however, the training of networks was not mentioned because the GPU performance is low in conditional processing but high in linear algebra processing. Therefore, we have proposed two methods. First, a whole network is implemented as a thread, and some networks are trained in parallel to shorten the time necessary to find the optimal weight coefficients. Second, this paper introduces parallelization in the neural network structure, in which the calculation of neurons in the same layers is parallelized. The processing to train the same network with different patterns is likewise independent. As a result, the second method is 20 times faster than the CPU, and 6 times as fast as the first proposed method. (c) 2013 Wiley Periodicals, Inc. Electron Comm Jpn, 96(8): 59-66, 2013; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.11412