Evaluating Performance, Power and Energy of Deep Neural Networks on CPUs and GPUs

Deep learning has achieved accuracy and fast training speed and has been successfully applied to many fields, including speech recognition, text processing, image processing and video processing. However, the cost of high power and energy comes together with the high accuracy and training speed of Deep Neural Network (DNN). This inspires researchers to perform characterization in terms of performance, power and energy for guiding the architecture design of DNN models. There are three critical issues to solve for designing a both accurate and energy-efficient DNN model: i) how the software parameters affect the DNN models; ii) how the hardware parameters affect the DNN models; and iii) how to choose the best energy-efficient DNN model. To answer the three issues above, we capture and analyze the performance, power and energy behaviors for multiple experiment settings. We evaluate four DNN models (i.e., LeNet, GoogLeNet, AlexNet, and CaffeNet) with various parameter settings (both hardware and software) on both CPU and GPU platforms. Evaluation results provide detailed DNN characterization and some key insights to facilitate the design of energy-efficient deep learning solutions.