Efficient memory reuse methodology for CNN-based real-time image processing in mobile-embedded systems

Real-time image processing applications such as intelligent security and traffic management requires pattern recognition tasks, such face recognition, and license plate detection, to execute in mobile-embedded systems. These mobile-embedded applications employ the deep neural network (DNN), especially convolutional neural network (CNN), to complete the image classification. However, deploying CNN models on embedded platforms is challenging as memory-costly CNNs are in conflict with the highly limited memory budget. To address this challenge, a variety of CNN memory reduction methodologies have been proposed. Among these methodologies, CNN memory reuse has no influence on accuracy and throughput of CNN and is easy to realize, which is most suitable for embedded application. However, the existing memory reuse algorithms cannot achieve stable optimal solution. To solve the problem, we first improve an existing memory reuse algorithm. Compared with its original version, the improved algorithm provides 7-25% less memory consumption of intermediate results. We further propose a novel CNN memory reuse algorithm. In the new algorithm, we significantly make use of CNN structure to reuse memory and obtain optimal solution at most cases. Compared with two existing memory reuse algorithms, the new algorithm can reduce the memory footprint by an average of 20.3% and 9.4%.