A multi-scale multi-level deep descriptor with saliency for image retrieval

In image retrieval, deep features from the fully-connected layer of Convolutional Neural Network(CNN) lack the ability to discriminate similar images containing small objects or structures. To address this problem, this work propose a multi-scale image representation with multiple semantic levels. The framework of our GOS (Global-Object-Salient) descriptor has three streams forming multiple feature levels. The global stream extracts features from the whole image of multiple resolutions and can capture multi-scale details. In the object stream, an object detector is used to detect objects of different scales at different locations. Object-level patches are of rectangular shape and may just contain a part of the object, so the salient stream is integrated to capture the most salient part of the image. GOS can capture global-level,object-level and salient-level features simultaneously. Experiments show that the three streams work in a complementary way and GOS framework is capable of producing competitive retrieval accuracy on four public image retrieval datasets.Specifically,we achieve 0.939(mAP),3.91(score@4), 0.908(mAP) respectively on Paris6K, UKB and Holidays dataset.