Parameter Servers Placement for Distributed Deep Learning in Edge computing

The architecture based on global-local parameter servers (PSs) is an appealing framework to accelerate the training of the global model in distributed deep learning. However, the accuracy and training time are influenced by the placement strategy of PSs in distributed deep learning, due to the adopted gradient descent methods and the heterogeneous computation and communication resources. Therefore, this paper formulates a novel problem for placement strategy of PSs in the dynamic available storage capacity, with the objective of minimizing the training time of the distributed deep learning under the constraints of the storage capacity and the number of local PSs. Then, we prove the NP-hardness of the proposed problem. To solve the problem, a parameter servers placement algorithm, followed by an adjustment algorithm, is proposed in this paper, by continuously making decisions for placement strategy of PSs to decrease the training time of the global model. Simulation results show that, the proposed combined algorithm performs better than existing works for all cases, in terms of the training time of global model. Moreover, the performance of the proposed algorithm is nearly close to that of brute-force approach, in terms of the training time of the global model.