Online Learning of Sparse Pseudo-Input Gaussian Process

In this paper, we propose a novel method of online learning of sparse pseudo-data, representative of the whole training data, for Gaussian Process (GP) regressions. We call the proposed method Incremental Sparse Pseudo-input Gaussian Process (ISPGP) regression. The proposed ISPGP algorithm allows for training from either a huge amount of training data by scanning through it only once or an online incremental training dataset. Thanks to the nature of the incremental learning algorithm, the proposed ISPGP algorithm can theoretically work with infinite data to which the conventional GP or SPGP algorithm is not applicable. From our experimental results on the KIN40K dataset, we can see that the proposed ISPGP algorithm is comparable to the conventional GP algorithm using the same number of training data. Although the proposed ISPGP algorithm performs slightly worse than Snelson and Ghahramani's SPGP algorithm, the level of performance degradation is acceptable.