Semi-supervised regression with manifold: A Bayesian deep kernel learning approach

Semi-supervised learning (SSL) aims at utilizing the vast unlabeled data to help the supervised training. While existing SSL methods have shown promising results on image classification tasks, most of them rely on the cluster assumption that does not apply to image regression tasks. In this paper, we address the under-studied semi-supervised image regression problem, of which the outputs are continuous val-ues instead of categorical distributions. To tackle this challenging task, we propose an algorithm, called ManiDKL, with the idea that the prediction function should be smooth with respect to the intrinsic man-ifold of data distribution and behave similarly on both labeled and unlabeled data. In particular, we pro -pose a framework that implements the Tikhonov regularization with generative manifold learning to ensure manifold smoothness of regression function and also reduces the problem to kernel learning. Then a semi-supervised non-parametric Bayesian based deep kernel learning algorithm is proposed, in which unlabeled data are incorporated through posterior regularization. We show the effectiveness of ManiDKL with extensive experiments. It shows that ManiDKL performs comparatively with state-of-the-art SSL image classification methods. Most importantly, we show the superiority of ManiDKL over all existing SSL regression methods on public image datasets. (c) 2022 Elsevier B.V. All rights reserved.