Image Super-Resolution by Neural Texture Transfer

Due to the significant information loss in low-resolution (LR) images, it has become extremely challenging tofurther advance the state-of-the-art of single image super-resolution (SISR). Reference-based super-resolution (ReJSR), on the other hand, has proven to be promising in recovering high-resolution (HR) details when a reference (Ref) image with similar content as that of the LR input is given. However, the quality of RejSR can degrade severely when Ref is less similar. This paper aims to unleash the potential of RefSR by leveraging more texture details from Ref images with stronger robustness even when irrelevantRef images are provided. Inspired by the recent work on image stylization, we formulate the ReJSR problem as neural texture transfer We design an end-to-end deep model which enriches HR details by adaptively transferringthe texturefrom Ref images accordingto their textural similarity. Instead of matching content in the rawpixel space as done by previous methods, ourkey contribution is a multi-level matching conducted in the neuralspace. This matchingschemefacilitates multi-scale neural transfer that allows the model to benefit more from those semantically related Ref patches, and gracefully degrade to SISR performance on the least relevant Ref inputs. We build a benchmark datasetfor the general research of RefSR, which contains Ref images paired with LR inputs with varying levels of similarity. Both quantitative and qualitative evaluations demonstrate the superiority of our method over state-of-the-art1