User-Driven Lossy Compression for Images and Video

In any given scene, a human observer is typically more interested in some objects than others, and will pay more attention to those objects they are interested in. This paper aims to capture this attention focusing behavior by selectively merging a fine-scale oversegmentation of a frame so that interesting regions are segmented into smaller regions than uninteresting regions. This results in a new type of image partitioning which reflects in the image the amount of attention we pay to a particular image region. This is done using a novel, interactive method for learning merging rules for images and videos based on defining a weighted distance metric between adjacent oversegments. We present as an example application of this technique a new lossy image and video stream compression method which attempts to minimize the loss in areas of interest.