Efficient binary 3D convolutional neural network and hardware accelerator

The three-dimensional convolutional neural networks have abundant parameters and computational costs. It is urgent to compress the three-dimensional convolutional neural network. In this paper, an efficient and simple binary three-dimensional convolutional neural network architecture is proposed, in which the weight and activation are constrained to 0 or 1 instead of the common + 1 or - 1. Binary weight and activation are first applied to the three-dimensional convolutional neural networks. The proposed binary three-dimensional convolutional neural network has less computational complexity and memory consumption than standard convolution, and it is more appropriate for digital hardware design. Furthermore, an optimized convolution operation is proposed, in which case one input pixel is only required to be read once. A distributed storage approach is proposed to support the proposed convolution operation. With the proposed methods, a hardware accelerator for the binary three-dimensional convolutional neural network on the field programmable gate array platform is designed. The experimental results show that the presented accelerator is excellent in terms of computational resources and power efficiency. By jointly optimizing the algorithm and hardware, the accelerator achieves 89.2% accuracy and 384 frames per second on the KTH dataset.