Collective, Hierarchical Clustering from distributed, heterogeneous data

This paper presents the Collective Hierarchical Clustering (CHC) algorithm for analyzing distributed, heterogeneous data, This algorithm first generates local cluster models and then combines them to generate the global cluster model of the data. The proposed algorithm runs in O(/S/n(2)) time, with a O(/S/n) space requirement and O(n) communication requirement, where n is the number of elements in the data set and ISI is the number of data sites. This approach shows significant improvement over naive methods with O(n(2)) communication costs in the case that the entire distance matrix is transmitted and O(nm) communication costs to centralize the data, where m is the total number of features. A specific implementation based on the single link clustering and results comparing its performance with that of a centralized clustering algorithm are presented. An analysis of the algorithm complexity, in terms of overall computation time and communication requirements, is presented.