A Proximal Gradient Algorithm for Composite Consensus Optimization over Directed Graphs

This paper proposes a decentralized algorithm for solving a consensus optimization problem defined in a directed networked multi-agent system, where the local objective functions have the smooth+nonsmooth composite form. Examples of such problems include decentralized compressed sensing and constrained quadratic programming problems, as well as many decentralized regularization problems. We extend the existing algorithms PG-EXTRA and ExtraPush to a new algorithm PG-ExtraPush for composite consensus optimization over a directed network. This algorithm takes advantage of the proximity operator like in PG-EXTRA to deal with the nonsmooth term, and employs the push-sum protocol like in ExtraPush to tackle the bias introduced by the directed network. We show that PG-ExtraPush converges to an optimal solution under the boundedness assumption. In numerical experiments, with a proper step size, PG-ExtraPush performs surprisingly linear rates, and is significantly faster than Subgradient-Push, even when we hand-optimize the step sizes for the latter.