Image inpainting based on patch-oriented gradient energy

This paper presents a new exemplar-based image inpainting algorithm for image inpainting and object removal. A concept, called patch-oriented gradient energy, is introduced to estimate the color change of an image patch. By means of the maximum patch-oriented gradient energy among all the patches, the direction and strength of an edge in the patch can be estimated. Exemplar-based inpainting methods often contain three steps: the first is to compute the priority of each patch to be inpainted; the second is to match the patches; and the third is to update the global parameters such as unknown region's contour and the confidence values. Using the oriented gradient energy in the normal direction of the unknown region's contour, we firstly select the patches containing the strongest structure information (i.e., edge information). In the matching step, we compare not only the color values of the corresponding pixels, but also the edge information in the patches by the patch-oriented gradient energy. The experiments show that our algorithm can propagate the structures and textures near boundaries of the areas to be inpainted to the unknown regions in a correct way to obtain results with visually smooth edges in the inpainted image.