Semantic Correspondence in the Wild

Semantic correspondence estimation where the object instances exhibits extreme deformations from one instance to the next is a challenging and important computer vision problem that needs to be solved. Unfortunately, all existing approaches require prior knowledge of the object classes which may not be available in the real world scenarios. This precludes the establishment of semantic correspondence across object classes in wild conditions when it is uncertain which classes will be of interest. In contrast, in this paper we formulate the semantic correspondence estimation task as a key point detection process in which image-to-class classification and image-to-image correspondence are solved simultaneously. Jointly modeling object identification and keypoint correspondence to attack the problem of semantic correspondence not only increases this approachs applicability in real world scenarios but also provides a seamless end-to-end process. Exploiting object regions in the process also enhances the accuracy while constraining the search space, thus improving overall efficiency. This new approach is compared with the state-of-the-art on public datasets including PF-PASCAL and Caltech-101 to validate its capability for improved semantic correspondence estimation in wild conditions.