Weighted ADMM for Fast Decentralized Network Optimization

In this paper, we propose a weighted alternating direction method of multipliers (ADMM) to solve the consensus optimization problem over a decentralized network. In the proposed algorithm, every node holds its local objective function, exchanges its current iterate with a subset of neighbors, carries on local computation, and eventually reaches an optimal and consensual solution that minimizes the summation of the local objective functions. Compared with the conventional ADMM that is popular in decentralized network optimization, the weighted ADMM is able to reduce the communication cost spent in the optimization process through tuning the weight matrices, which assign beliefs on the neighboring iterates. We first prove convergence and establish linear convergence rate of the weighted ADMM. Second, we maximize the derived convergence speed and obtain the best weight matrices on a given topology. Third, observing that exchanging information with all the neighbors is expensive, we maximize the convergence speed while limit the number of communication arcs. This strategy finds a subset of arcs within the underlying topology to fulfill the optimization task while leads to a favorable tradeoff between the number of iterations and the communication cost per iteration. Numerical experiments demonstrate advantages of the weighted ADMM over its conventional counterpart in expediting the convergence speed and reducing the communication cost.