Nonlocal-guided enhanced interaction spatial-temporal network for compressed video super-resolution

Although deep-learning based video super-resolution (VSR) studies have achieved excellent progress in recent years, the majority of them do not take into account the impact of lossy compression. A large number of real-world videos are characterized by compression artifacts (e.g., blocking, ringing, and blurring) due to transmission bandwidth or storage capacity limitations, which makes the VSR task more challenging. To balance compression artifacts reduction and detail preservation, this paper proposes a nonlocal-guided enhanced interaction spatial-temporal network for compressed video super-resolution (EISTNet). EISTNet consists of the nonlocal-guided enhanced interaction feature extraction module (EIFEM) and the attention-based multi-channel feature self-calibration module (MCFSM). The pixel-shuffle-based nonlocal feature guidance module (PNFGM) is designed to explore the nonlocal similarity of video sequences and then it is used to guide the extraction and fusion of inter-frame spatial-temporal information in EIFEM. Considering that compression noise and video content are strongly correlated, MCFSM introduces features from the compression artifacts reduction stage for recalibration and adaptive fusion, which closely associates the two parts of the network. To reduce the computational memory pressure on nonlocal modules, we add pixel-shuffle operation to PNFGM, which also expands its receptive field. Experimental results demonstrate that our method achieves better performance compared to the existing methods.