CNN-based Bi-prediction Utilizing Spatial Information for Video Coding

In video coding, slice-level and block-level weighted bi-prediction are used for scenes with temporal brightness variation. However, there are still structured residuals when applying weighted bi-prediction in slice and block level. Recently, CNN-based bi-prediction has achieved remarkable success on reducing significant structured residuals, in which bi-predictor is generated by CNN model using two reference blocks as inputs. Inspired by high spatial correlation of pixels, this paper uses spatial neighboring pixels of both current block and two reference blocks as the additional information of the proposed CNN model to further reduce residual and generate a more accurate bi-predictor. Moreover, by comparing AMVP and merge/skip mode, this paper illustrates that CNN-based bi-prediction is more efficient for merge/skip mode than for AMVP mode. Experimental results show that proposed method reaches 3.46% BD-rate saving for random access configuration on average compared to HM 16.15.