Multi-kernel Partial Least Squares for Multi-Modal Data Analysis

In recent years, multi-modal data analysis has enjoyed an increasing attention. Multi-modal data mean the different modal data representing the same semantics. Moreover, many subspace learning methods are proposed to measure the correlation between different modal data. As the most representative subspace learning method, canonical correlation analysis (CCA) and its variants project different modal data into a common space where the Pearson correlation is maximized. Yet CCA often causes information loss when switching the modals, and as a result, the partial least squares regression (PLSR) model is adopted to handle the problem. Subsequently, considering the nonlinearity of data, the kernel partial least squares regression (KPLSR) is proposed. Besides, KPLSR mostly relies on the kernel parameters. Hence, we propose to apply multi-kernel partial least squares regression (MKPLSR) for multi-modal data analysis. To evaluate the proposed approach, massive experiments are carried out. Compared with previous methods, the experimental results on two benchmark datasets composed of images and texts pairs, show the effectiveness of our approach, when applied to multi-modal data retrieval and classification.