Spatial-Temporal Inter-Layer Reference Frame Generation Network for Spatial SHVC

In the current spatial Scalable High Efficiency Video Coding (SHVC) standard, the main techniques involve exploiting the correlation between pixel values of different layers to achieve inter-layer prediction samples, allowing the enhancement layer (EL) to predict samples from the upsampled base layer (BL) frame and remove temporal redundancy. However, existing network-based methods cannot effectively handle multi-layer compressed images with different resolutions to generate reference frame in spatial SHVC. Meanwhile, spatial SHVC only uses traditional interpolation filters to upsample the BL frame for EL frame sample prediction, which cannot handle different structures and contents. Therefore, considering the high correlation of multi-scale distortion characteristics across different layers, this article proposes a spatial-temporal inter-layer reference frame generation network (ST-ILR) for spatial SHVC, which can generate a high-fidelity reference frame for efficient inter-prediction and insert it into the EL reference picture list. The proposed method consists of two modules: a multi-scale motion restoration (MMR) module and a guided multi-scale feature reconstruction (GMFR) module. The MMR model is designed to accurately predict the motion trend of the EL based on the BL motion information, while implicitly compensating for previous EL frames. This is achieved by dynamically modeling the current EL motion information from the BL, capturing compression downsampling differences of prior motion vectors across different layers. The GMFR module adaptively super-resolves compressed BL frames and selectively aggregates high-frequency information from aligned EL features to preserve precise spatial detail, fusing abundant features from different layers to achieve better ILR frame quality performance. Extensive experiments show that our network achieves a 13.6% BD-rate (Bjontegaard Delta Rate) reduction in random access configuration compared to the SHVC baseline, which offers state-of-the-art coding performance.