Towards Energy Efficient DNN accelerator via Sparsified Gradual Knowledge Distillation

Artificial intelligence (AI) is becoming increasingly popular in many applications. However, the computation cost of deep neural network (DNN) , which is a powerful form of AI, calls for efficient DNN compression technique to make energy efficient networks. In this paper, we proposed SKG, a method to jointly sparsify and quantize DNN models to ultra-low bit-precision using Knowledge Distillation and gradual quantization (SKG). We demonstrated that our method can preserve the accuracy more than 20% for uniform quantization with 2 bit-width compared to the baseline methods on ImageNet and ResNet-18. In addition, our method can achieve up to 2.7x lower energy consumption using compute-in-memory (CIM) architecture compared to a traditional 65nm CMOS architecture for both pruned and unpruned network during inference and eventually enabling using DNN models on resource constrained edge devices.