Unsupervised image categorization and object localization using topic models and correspondences between images

Topic models from the text understanding literature have shown promising results in unsupervised image categorization and object localization. Categories are treated as topics, and words are formed by vector quantizing local descriptors of image patches. Limitations of topic models include their weakness in localizing objects, and the requirement of a fairly large proportion of words coming from the object. We present a new approach that employs correspondences between images to provide information about object configuration, which in turn enhances the reliability of object localization and categorization. This approach is efficient, as it requires only a small number of correspondences. We show improved categorization and localization performance on real and synthetic data. Moreover, we can push the limits of topic models when the proportion of words coming from the object is very low.