Soft Kernel Target Alignment for Two-Stage Multiple Kernel Learning

The two-stage multiple kernel learning (MKL) algorithms gained the popularity due to their simplicity and modularity. In this paper, we focus on two recently proposed two-stage MKL algorithms: ALIGNF and TSMKL. We first show through a simple vectorization of the input and target kernels that ALIGNF corresponds to a non-negative least squares and TSMKL to a non-negative SVM in the transformed space. Then we propose ALIGNF+, a soft version of ALIGNF, based on the observation that the dual problem of ALIGNF is essentially a one-class SVM problem. It turns out that the ALIGNF+ just requires an upper bound on the kernel weights of original ALIGNF. This upper bound makes ALIGNF+ interpolate between ALIGNF and the uniform combination of kernels. Our experiments demonstrate favorable performance and improved robustness of ALIGNF+ comparing to ALIGNF. Experiments data and code written in python are freely available at github (https://github.com/aalto-ics-kepaco/softALIGNF).