Patch-based self-adaptive matting for high-resolution image and video

We propose an efficient patch-based self-adaptive matting approach to reduce memory consumption in processing high-resolution image and video. Most existing image matting techniques employ a global optimization over the whole set of image pixels, incurring a prohibitively high memory consumption, especially in high-resolution images. Inspired by “divide-and-conquer,” we divide the images into small patches in a self-adaptive way according to the distribution of unknown pixels and handle the small patches one by one. The alpha mattes in patch level are combined according to the weights. Relationships between patches are also considered by locally linear embedding to maintain consistency through the whole image. We also extend the framework to video matting with considering the temporal coherence of alpha mattes. A sampling method is applied to speed up the operation of video sampling. A multi-frame graph model is also proposed to enhance temporal and spatial consistency which can be solved efficiently by Random Walk. Experimental results show that the proposed method significantly reduces memory consumption while maintaining high-fidelity matting results on the benchmark dataset.