Hashing for Cross-Modal Similarity Retrieval

Now, cross-modal retrieval similarity on multimedia with texts and images have attracted scholars' more and more attention. The difficulty of cross-modal retrieval is how to effectively construct correlation between multi-modal heterogeneous data. According to canonical correlation analysis, most existing cross-modal methods embed the heterogeneous data into a joint abstraction space by linear projections. The recognition accuracy is an urgent problem. To address this challenge, in this paper, we propose a adaptive boosting method with weighted CCA and Hash to solve cross-modal similarity retrieval. We use hash method to speed up the retrieval efficiency. Using the CCA to connect the image and text data with the original features. The weight is the input parameter of adaboost algorithm. We choose this algorithm to change the weight for reduce the CCA mapping error rate. We first capture the text component which is represented as a sample from a hidden topic model, learned with latent dirichlet allocation, and images are represented as bags of visual (SIFT and GIST) features. Second, the unified hash codes are generated through the high level abstraction space by hash method such as spectral hashing, kernelized locality semantic hashing and iterative quantization. Third, correlations between the two components are learned with weighted canonical correlation analysis. In this part, we use the Adaboost to iterate this process to find the best result. Finally, find out the nearest neighbors. Extensive experiments on two different datasets highlight the advantage of our method.