Coarse Label Refined Knowledge Reasoning for Fine-Grained Visual Categorization

Fine-grained visual categorization (FGVC) is a challenging task due to large intra-class variance and small inter-class variance, aiming at recognizing hundreds of subcategories which belong to the same basic-level category. Humans have the ability to acquire knowledge about the world and use the acquired knowledge to reason about entities. When identifying an image, humans first reason out candidate subcategories of the object based on their prior knowledge. Then discover the characteristics of the object, which can be used as assistance for comparisons between candidate subcategories. Finally humans can determine the ultimate prediction. Inspired by this behavior, we propose a Coarse Label Refined Knowledge Reasoning (CLRKR) approach for fine-grained visual categorization. Its main novelties and advantages are as follows: (1) Knowledge Reasoning: We first construct the knowledge graph based on subcategory-attribute correlations, then reason out candidate subcategories with corresponding probabilities based on the knowledge graph. (2) Coarse Label Refinement: We incorporate coarse classes, which are pre-defined groups of attributes, to get corresponding coarse labels. These coarse labels reveal the most characteristic features of the entity in the image, which help us to re-rank the sequence of candidate subcategories and acquire the foremost subcategory as the final prediction. Our CLRKR approach achieves the best performance according to the experimental results on the widely-used CUB-200-2011 dataset for fine-grained visual categorization comparing with the state-of-the-art methods.