Optimizing Convolutional Neural Network on FPGA under Heterogeneous Computing Framework with OpenCL

As convolutional neural network (CNN) has been used more and more widely, such as in areas of images classifications and face recognition, the traditional CPU or GPU platforms have been insufficient to support the efficient operation of increasingly complex CNN. Therefore, heterogeneous computing platform is increasingly used to accelerate CNN, which contains a host and one or more computing devices, such as GPU and FPGA, etc. Due to its programmable hardware structure and high power efficient, FPGA is very promising in CNN acceleration. OpenCL is designed to provide a unified framework of heterogeneous computing platform for the industry. As FPGA vendors gradually began to support OpenCL, it is possible to use OpenCL on FPGA, which makes the development on FPGA much easier. This paper uses Xilinx SDAccel tool to explore how to accelerate CNN in OpenCL framework with FPGA, especially for CNN applications. Since convolutional layer is the most complex part of CNN, this paper first focuses on optimizations of a single convolution layer, then discusses the acceleration of a complete CNN where different optimization strategies are being explored. By using appropriate optimization measures, the computing speed of CNN on FPGA has improved. For convolutional layer, an improvement of speedup of 14.4X has been achieved. Moreover, the processing speed of some CNN network has been improved 2X with pipelined structure as well as the overall throughput is 48.5fps. Additionally, the utilization of the hardware resources of FPGA chip is less than 30%, which means the use of OpenCL on FPGA to accelerate CNN would be of great prospects with increasingly sophisticated tool chain, supporting FPGA hardware improvements and appropriate optimizations of CNN algorithms.