Multi-frame spatio-temporal super-resolution

Increasing the resolution of digital images and videos using digital super-resolution (SR) techniques has been of great interest in industry and academia over the past three decades. Most SR methods target improving only the spatial resolution of images and videos, whereas improving the temporal resolution could be more critical for some videos. Motion blur is a temporal artifact by nature, so removing it using spatial SR techniques would be highly challenging and often unsuccessful. This paper proposes a multi-frame motion-based video super-resolution method to increase both spatial and temporal resolutions of a single input video. Our optimization problem is based on a maximum a posteriori estimator that estimates each high-resolution (HR) frame by fusing multiple low-resolution frames. The form of the image prior used in the optimization framework is based on the assumption that natural HR frames are piecewise smooth. We introduce a new method to enhance the sharpness of edges in the video frames during the optimization process. We also involve a temporal constraint that improves temporal consistency in the estimated video. Moreover, we propose a new scheme for motion estimation that better suits video frame rate upsampling. Our results are compared with state-of-the-art SR methods, including ML-based ones, which confirm the effectiveness of the proposed method.