Deep Reference Frame Interpolation based Inter Prediction Enhancement for Versatile Video Coding

In video coding, bi-directional inter prediction aims to remove temporal redundancy via two-sided previously coded frames as reference. High-quality reference frames are essential to reduce the prediction residuals and improve coding efficiency performance. In this paper, we propose a deep learning-based reference frame interpolation method to enhance bi-prediction by introducing a synthetic frame to reference picture lists. Specifically, reconstructed frames are fed into a well-designed interpolation and filtering network to synthesize a picture which can be regarded as an additional reference of to-be-coded frame. Then, the picture is inserted at the appropriate place in reference picture lists to provide a more reliable reference for subsequent motion estimation and motion compensation. Experimental results show that the proposed method achieves 2.03%/6.96%/6.40% coding efficiency improvements for Y/U/V components under random access configuration, when compared with the VVC reference software VTM-15.0.