An Energy-Efficient Deep Learning Processor with Heterogeneous Multi-Core Architecture for Convolutional Neural Networks and Recurrent Neural Networks

An energy-efficient deep learning processor is proposed for convolutional neural networks (CNNs) and recurrent neural networks (RNNs) in mobile platforms. The 16mm(2) chip is fabricated using 65nm technology with 3 key features, 1) Reconfigurable heterogeneous architecture to support both CNNs and RNNs, 2) LUT-based reconfigurable multiplier optimized for dynamic fixed-point with the on-line adaptation, 3) Quantization table-based matrix multiplication to reduce off-chip memory access and remove duplicated multiplications. As a result, compared to the [2] and [3], this work shows 20x and 4.5x higher energy efficiency, respectively. Also, DNPU shows 6.5(x) higher energy efficiency compared to the [5].