Deep Embedded Clustering with Random Projection Penalty

Despite being studied for decades, current clustering approaches still face difficulties in handling high-dimensional datasets. For high-dimensional data, it is essential to get a good feature representation for the clustering algorithm to conduct on. Most of the clustering algorithms do not explicitly encourage the preservation of pairwise distance within the input data when learning feature representation. We proposed an improvement over the Deep Embedded Clustering (DEC) by including a penalty term in the loss function for the differences between the input data and the random projection of its corresponding feature embedding. The idea behind this penalty term is one of the properties of random projection, that is the pairwise distance is preserved between the low-dimensional manifold and the data space. In this way, the network encourages the learning towards preserving data similarities in the feature space. We named the proposed method as DEC-RPP. The experiments show significant improvements on clustering metrics over four datasets compared with the baseline DEC and another work IDEC that improves DEC by preserving local structure.