An object- and user-driven system for semantic-based image annotation and retrieval

In this paper, a system for object-based semi-automatic indexing and retrieval of natural images is introduced. Three important concepts underpin the proposed system: a new strategy to fuse different low-level content descriptions; a learning technique involving user relevance feedback; and a novel object based model to link semantic terms and visual objects. To achieve high accuracy in the retrieval and subsequent annotation processes several low-level image primitives are combined in a suitable multifeatures space. This space is modelled in a structured way exploiting both low-level features and spatial contextual relations of image blocks. Support vector machines are used to learn from gathered information through relevance feedback. An adaptive convolution kernel is defined to handle the proposed structured multifeature space. The positive definite property of the introduced kernel is proven, as essential condition for uniqueness and optimality of the convex optimization in support vector machines. The proposed system has been thoroughly evaluated and selected results are reported in this paper.