Robust multilayer bootstrap networks in ensemble for unsupervised representation learning and clustering

It is known that unsupervised nonlinear learning is sensitive to the selection of hyperparameters, which hinders its practical use. How to determine the optimal hyperparameter setting that may be dramatically different across applications is a hard issue. In this paper, we aim to address this issue for multilayer bootstrap networks (MBN), a recent unsupervised model, in a way as simple as possible. Specifically, we first propose an MBN ensemble (MBN-E) algorithm which concatenates the sparse outputs of a set of MBN base models with different network structures into a new representation. Then, we take the new representation produced by MBN-E as a reference for selecting the optimal MBN base models. Moreover, we propose a fast version of MBN-E (fMBN-E), which is not only theoretically even faster than a single standard MBN but also does not increase the estimation error of MBN-E. Empirically, comparing to a number of advanced clustering methods, the proposed methods reach reasonable performance in their default settings. fMBN-E is empirically hundreds of times faster than MBN-E without suffering performance degradation. The applications to image segmentation and graph data mining further demonstrate the advantage of the proposed methods.