Product image retrieval using category-aware siamese convolutional neural network feature

Product image retrieval in the customer-to-shop setting uses similarity learning instead of a predefined distance to address the cross-domain matching problem. Similarity learning can be done using a Siamese convolutional network (SCN) model with pairwise or triplet image sampling. The model training uses product item labels as the target without considering the product category. However, images in the eshop are inherently have hierarchically structured from the category to the individual image. Therefore, category information should be involved to improve the discriminating factor of the image feature. To accommodate this, we propose a SCN model that involves category and item labels in training to produce the category-aware feature. Our model is based on SCN with modification in training procedure that simultaneously learns the category and item label. Our category-aware Siamese CNN is implemented using MobileNet as the backbone and single-layer network for the mid-feature learner. The results show that our method can improve the accuracy of product image retrieval using SCN based features. (c) 2022 The Authors. Published by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).