Exploiting Semantic Structured Relationships Using Graph Models for Semantic Annotations

This paper proposes a novel and efficient approach to exploit semantic relationships using semantic modeling for semantic annotation tasks. The existing methods learn knowledge of concepts and their relationships based on context cues. Starting with a large set of objects detectors, the proposed method refines the initial annotation results using the learned semantic relationships, which can preserve the consistency and effective of the annotation over a semantic graph. Different from the existing graph learning methods which capture relations among data instances, the semantic graphs treat concepts as nodes and concept affinities as the weights of edges. Particularly, the proposed method can not only learn the semantic cues effectively through the semantic graph models to improve the annotation results, but also can adapt the concept affinities to unseen images. The method provides a means to handle structured relationship change between training and test data, which occurs very often in semantic annotation tasks. Our experiments on NYUv2 demonstrate that the proposed approach outperform the state-of-the-art algorithms.