Spark-based Parallel Cooperative Co-evolution Particle Swarm Optimization Algorithm

Traditional particle swarm optimization algorithms (PSO) targeted to solve large scale problems are mostly serial, such as CCPSO2, and the computing time is very long in general. Therefore, this paper presents a novel parallel PSO, which explores the usage of new probability distribution functions for the replacement of traditional Gaussian and Cauchy distributions, and the combination of GPSO and LPSO to make use of space exploration and speed up the convergence. As to the implementation of algorithm parallelization, we adopt the Spark platform, which is one of the currently most popular big data processing tools. We make modification to dynamic grouping and multiple calculations, in order to increase the degree of parallelism, reduce the computation time and improve algorithm efficiency as far as possible. Multiple computing refers to that in each single distribution of tasks, one computing node processes the particle position information of multiple algorithms. In the control of space exploration and convergence rate, we present a more efficient method to explore the solution space, which controls the convergence rate to enhance the exploration to a greater extent and also ensures fast convergence rate at the later stage, thus, it not only guarantees the calculation speed, but also improves the optimization effect as more as possible. We used twenty LSGO benchmark functions in CEC'2010 to make experiments, showing that the proposed algorithm could obtain satisfactory results, and for some functions, it outperforms DECC and MLCC.