Classification-based multimodality fusion approach for similarity ranking

The need for similarity rankings is common to a wide diversity of Pattern Recognition problems. When multiple modalities are available, effective combination methods that exploit the information contained in the different representations are required. In this paper, a method for effectively combining the information in the different modalities is presented. The method adopts the common framework used in metric learning and assumes that training samples are available, in the form of pairs of objects labeled as similar or dissimilar. For each pair, one or more distance measures are computed in each representation space, and these are used to train a soft classifier. Estimated class conditional probabilities are then used as scores for ranking purposes. The approach has been tested and compared to other existing combination methods in an image retrieval context, showing competitive results.