Deep learning-based intra prediction mode decision for HEVC

The High Efficiency Video Coding standard and its screen content coding extension provide superior coding efficiency compared to predecessor standards. However, this coding efficiency is achieved at the expense of very complex encoders. One major complexity driver is the comprehensive rate distortion (RD) optimization. In this paper, we present a deep learning-based encoder control which replaces the conventional RD optimization for the intra prediction mode with deep convolutional neural network (CNN) classifiers. Thereby, we save the RD optimization complexity. Our classifiers operate independently of any encoder decisions and reconstructed sample values. Thus, no additional systematic latency is introduced. Furthermore, the loss in coding efficiency is negligible with an average value of 0.52% over HM-16.6+SCM-5.2.