From Text to Images: Weighting Schemes for Image Retrieval

Bags of visual words are the most studied image description technique in the last years. This representation of images raised new possibilities as well as new research issues. In particular, it is important to automatically determine which visual words are the most relevant to describe the images, and which ones should be ignored. This issue is a classical problem of textual information retrieval, usually addressed by the use of weighting schemes. In this paper, the most common weighting schemes from text retrieval are applied to the case of visual word-based retrieval. New weighting schemes are also proposed, and several Minkowski-like distances are tested. The experiments are performed on four different datasets that correspond to two different retrieval tasks; it allows us to bring to light some properties of visual words and weighting schemes. This study results in several findings. It first shows that the optimal setting for distances and weighting schemes depends on the nature of the visual content of the images considered. Especially, raw frequency can be the most effective weight when dealing with complex datasets; it questions the habit to systematically use the tf . idf weighting scheme. It also shows that weighting schemes and Minkowski distances have similar effect and should be used together in a consistent way. Based on these findings, general guidelines for the choice of distances and weighting schemes are proposed.