On Vocabulary Size in Bag-of-Visual-Words Representation

Bag-of-visual-words is a popular image representation that produces high matching accuracy and efficiency. While vocabulary size impacts on matching accuracy, existing research usually selects the vocabulary size empirically. Research on representative local descriptors shows that with similarity based clustering, the intra-cluster similarity extent of descriptors plays the same role in straightforward matching as vocabulary size in visual words matching. Based on this observation, we propose to use similarity based clustering to determine the optimal vocabulary size for a given dataset in visual words matching. Preliminary experiments with three datasets produce encouraging results and demonstrate the potential of the proposed approach.