Research on SCKM Algorithm Based on the Parallel Clustering

As an effective method in dealing with the massive data, the serial processing, aiming to obtain the useful information quickly, cannot satisfy our calculation requirements with high-performance. However, both distributed computing and parallel computing are good choices in calculating high-volume data with high-performance. As a parallel computing framework based on memory computing large data, Spark was of great concern as soon as it was proposed. This paper, based on the limitations of k-medoids algorithm which is sensitive to the center point and requires a large number of iterations in the process of calculating the new center point, puts forward a parallel algorithm named Canopy-Kmedoids within the platform of Spark. The algorithm obtains the K center points by the Canopy algorithm. By analyzing the performance of parallel operators and the advantage of using Spark to have good performance for iterative computation. In addition, this method reduces the frequency of reading or writing the shuffle and disk, which effectively overcomes the shortcomings of k-medoids. The experimental results show that the parallel algorithm achieves a relative ideal speedup ratio and can handle the massive data efficiently as well.