Design of high performance and energy efficient convolution array for convolution neural network-based image inference engine

The energy efficiency of CNN-based inference engines predominately depends upon Giga-operations-per-second and power consumption. The sparse-based accelerator compresses the insignificant inputs (input feature maps & weights), skips the inefficient computations, and improves energy efficiency. A sparse accelerator for weights could impact the accuracy of the inference. Therefore, a sparse network for Input Feature Maps (IFMs) is considered. MATLAB-based sparsity analysis is done layer-wise on the pre-trained CNN models like AlexNet, VGG-16, VGG-19, ResNet-18 & ResNet-34. Layer-wise analysis reveals that similar to 18%-90% of the IFMs are zeros. Besides, IFMs and Weights adopted the 16-bit Fix/Float data format to maintain an accuracy as close as 97% with Single Precision Floating Point (SPFP). A 3 x 1 Convolutional array with improved Zero-detect-Skip (CZS(3x1)) control units for multiplier and adder/subtractor arrays is proposed. The modified rectified linear unit (RELU) converts IFM values <= 0 to zero and sets Detection-Bit (DB) to 1. These DBs decide the mode of effective zero-skip operations in CZS(3x1). A 3 x 3 Compressed Processing Element (CPE) is designed using three CZS(3x1). The 20-CPEs convolution array architecture is implemented in 65 nm technology libraries. The performance of 90 Giga-operations per second (GOP/s) and energy efficiency of 3.42 Tera operations per second per watt (TOPS/W) were attained for the proposed CPE. The CPE with improved control strategy enhanced the performance by a factor of 2.45 while consuming 8.8 times less energy on average than the state-of-the-art CNN accelerators.