Distance Penalization Embedding for Unsupervised Dimensionality Reduction

Local structures and global structures of the data set are both important information for learning from data. However, most manifold learning algorithms, such as LLE,Laplacian Eigenmap, LTSA, et.al paid great attention to preserving the local structures of data set, but neglected the global structure of the data set. ISOMap considers both the local structures and global structures; however, the constraint of preserving the global manifold distances is so strict that ISOMap would fail on some manifolds that cannot isometrically map to a lower dimensional Euclidean space. In this paper, we proposed a new method - Distance Penalization Embedding, which preserves the global structures of data sets in a more flexible way under the constraint of local structure preserving. Experimental results on the data sets with high nonlinearity show good performances of the proposed method.