An Efficient Data Distribution Strategy for Distributed Graph Processing System

Big data applications like social networks, biological networks, etc. are often realized on graphs. Graph processing, if done on a single node, increases time complexity. Partitioning of graphs has been proved to be useful towards handle this well-known issue. There are several partitioning algorithms that are used to partition a graph. Each partition is assigned to a node within a cluster. However, the storage capacity of a node is limited. Therefore, an effective data distribution mechanism is required. This work aims to propose a novel strategy that would define an efficient distribution of graphs into nodes using genetic algorithms. The proposed data distribution strategy, when applied on two benchmark data set, shows improved data availability without increasing the number of replicas. It has also observed that the execution time will almost became half after applying the proposed method.