From Universal Bag-of-Words to Adaptive Bag-of-Phrases for Mobile Scene Recognition

This paper proposes an adaptive bag-of-phrases (BoP) algorithm for mobile scene recognition based on bag-of-words approach. Conventional BoW methods do not consider the dependence and pairwise relationship among different codewords. However, these contextual relations between pairwise codewords play an important role for users to recognize an image. In light of this problem, this paper proposes an effective BoP technique to integrate both the spatial and contextual information between visual words for scene recognition. It first uses hierarchical k-means algorithm to construct a universal codebook for all categories. The contextual (dependence) relationship between pairwise words is then mined for each category based on the mutual information they contain. Subsequently, a visual phrase vocabulary is constructed which is then used to generate a BoP histogram through a proposed quantization method. Finally, support vector machine (SVM) is used to train these histograms into a classifier. Experimental results on the Scene 15 dataset show that the proposed method is effective for mobile scene recognition.