Computationally efficient image restoration and super-resolution algorithms for real-time implementation

Computational complexity is a major impediment to the real-time implementation of image restoration and super-resolution algorithms. Although powerful restoration algorithms have been developed within the last few years utilizing sophisticated mathematical machinery (based on statistical optimization and convex set theory), these algorithms are typically iterative in nature and require enough number of iterations to be executed to achieve desired resolution gains in order to meaningfully perform detection and recognition tasks in practice. Additionally, recent technological breakthroughs have facilitated novel sensor designs (focal plane arrays, for instance) that make it possible to capture mega-pixel imagery data at video frame rates. A major challenge in the processing of these large format images is to complete the execution of the image processing steps within the frame capture times and to keep up with the output rate of the sensor so that all data captured by the sensor can be efficiently utilized. Consequently, development of novel methods that facilitate real-time implementation of image restoration and super-resolution algorithms is of significant practical interest and will be the primary focus of this paper. The key to designing computationally efficient processing schemes lies in strategically introducing appropriate pre-processing and post-processing steps together with the super-resolution iterations in order to tailor optimized overall processing sequences for imagery data of specific formats. Three distinct methods for tailoring a pre-processing filter and integrating it with the super-resolution processing steps will be outlined in this paper. These methods consist of a Region-of-Interest (ROI) extraction scheme, a background-detail separation procedure, and a scene-derived information extraction step for implementing a set-theoretic restoration of the image that is less demanding in computation compared to the super-resolution iterations. A quantitative evaluation of the performance of these algorithms for restoring and super-resolving tactical imagery data, including in particular PMMW images, will also be presented.