A hierarchical sampling based triplet network for fine-grained image classification

Deep metric learning leverages well-designed distance measurement and a sample selection strategy to learn a discriminative feature space. Among the various deep metric learning formulations, triplet loss is built based on a 3-tuple that can simultaneously minimise the distance between the items in the positive pair and maximise the distance between those in the negative pair. However, this endeavour requires a critical selection of triplet samples to guide the training process. In this paper, we propose a layered Triplet loss to solve the fine-grained image classification problem. Unlike the existing triplet loss, which selects samples from only a single criterion, we construct the loss function with the 'coarse to fine' scheme. This scheme can separate the coarse-level classes while clustering the fine-level samples within a certain margin. An ontology-based sampling method is proposed to enable the network to mine more reasonable hard triplets. Semantic knowledge is employed to assign the visually similar classes to the same learning task, from which hard triplets can be generated. Finally, the softmax tree classifier is used to classify the hierarchical features. The experimental results on multiple datasets demonstrate the effectiveness of the proposed method. (c) 2021 Elsevier Ltd. All rights reserved.