Evaluating Data Terms for Variational Multi-frame Super-Resolution

We present the first systematic evaluation of the data terms for multi-frame super-resolution within a variational model. The various data terms are derived by permuting the order of the blur-, downsample-, and warp-operators in the image acquisition model. This yields six different basic models. Our experiments using synthetic images with known ground truth show that two models are preferable: the widely-used warp-blur-downsample model that is physically plausible if atmospheric blur is negligible, and the hardly considered blur-warp-downsample model. We show that the quality of motion estimation plays the decisive role on which of these two models works best: While the classic warp-blur-downsample model requires optimal motion estimation, the rarely-used blur-warp-downsample model should be preferred in practically relevant scenarios when motion estimation is suboptimal. This confirms a widely ignored result by Wang and Qi (2004). Last but not least, we propose a new modification of the blur-warp-downsample model that offers a very significant speed-up without substantial loss in the reconstruction quality.