Introducing an Atypical Loss: A Perceptual Metric Learning for Image Pairing

Recent works have shown an interest in comparing visually similar but semantically different instances. The paired Totally Looks Like (TLL) image dataset is a good example of visually similar paired images to figure out how humans compare images. In this research, we consider these more generic annotated categories to build a semantic manifold distance. We introduce an atypical triplet-loss using the inverse Kullback-Leibler divergence to model the distribution of the anchor-positive (a-p) distances. In the new redefinition of triplet-loss, the anchor-negative (a-n) loss is conditional to the a-p distance distribution which prevents the loss correction fluctuations in the plain summed triplet-loss function of absolute distances. The proposed atypical triplet-loss builds a manifold from relative distances to a "super" anchor represented by the a-p distribution. The evaluation on the paired images of the TLL dataset showed that the retrieving score from the first candidate guess (top-1) is 75% which is x 2.5 higher compared to the recall score of the baseline triplet-loss which is limited to 29%, and with a top-5 pairing score as high as 78% which represents a gain of x1.4.