Information Fusion for Combining Visual and Textual Image Retrieval in ImageCLEF@ICPR

In the ImageCLEF image retrieval competition multimodal image retrieval has been evaluated over the past seven years. For ICPR 2010 a contest was organized for the fusion of visual and textual retrieval as this was one task where most participants had problems. In this paper, classical approaches such as the maximum combinations (combMAX), the sum combinations (combSUM) and the multiplication of the sum and the number of non-zero scores (combMNZ) were employed and the trade-off between two fusion effects (chorus and dark horse effects) was studied based on the sum of n maxima. Various normalization strategies were tried out. The fusion algorithms are evaluated using the best four visual and textual runs of the ImageCLEF medical image retrieval task 2008 and 2009. The results show that fused runs outperform the best original runs and multi-modality fusion statistically outperforms single modality fusion. The logarithmic rank penalization shows to be the most stable normalization. The dark horse effect is in competition with the chorus effect and each of them can produce best fusion performance depending on the nature of the input data.