Nonlinear skeletons of data sets and applications - Methods based on subspace clustering

In many practical problems the data (given by the columns of a data matrix) lie, up to some precision, on a union of m affine subspaces, called affine skeleton of the data. Finding the best possible skeleton when m = 1 can be performed easily by Principal Component Analysis (PCA). When m > 1, however, this is a global optimization problem, which appears to be hard for bigger m. Its solution reveals an internal structure of the data set, namely, under mild assumptions, the data matrix can be decomposed uniquely (up to some unimportant ambiguities) as multiplication of a mixing matrix and a source matrix, which, more importantly, is sparse. This a kind of Blind Signal Separation problem based on identifiability conditions involving sparseness, which we describe precisely. We develop a subspace clustering algorithm, which is a generalization of the k-plane clustering algorithm, and is suitable for separation of sparse mixtures with bigger sparsity. We present "kernelization" of this idea, leading to the notion of nonlinear Skeletons, when we work in Reproducing Kernel Hilbert Spaces (RKHS). Nonlinear skeleton identification in RKHS is a fusion of two ideas: clustering and Kernel PCA. We propose the idea that binary classification tasks can be performed by finding nonlinear skeletons of the training points in the both classes. We demonstrate our algorithms by examples.