On Replica Symmetry Breaking in Vector Precoding for the Gaussian MIMO Broadcast Channel

The so-called "replica method" of statistical physics is employed for the large system analysis of vector precoding for the Gaussian multiple-input multiple-output (MIMO) broadcast channel. Focusing on discrete complex input alphabets, the transmitter is assumed to comprise a linear front-end combined with nonlinear precoding, that minimizes the front-end imposed transmit energy penalty. The energy penalty is minimized by relaxing the input alphabet to a larger alphabet set prior to precoding. The limiting empirical distribution of the precoder's output, as well as the limiting energy penalty, are derived while harnessing what is referred to as the first order replica symmetry breaking (1RSB) ansatz. Particularizing to a "zero-forcing" (ZF) linear front-end, and non-cooperative users, a decoupling result is derived according to which the channel observed by each of the individual receivers can be effectively characterized by the Markov chain u-x-y, where u is the channel input, X is the equivalent precoder output, and y is the channel output. An illustrative example is considered, based on discrete-lattice alphabet relaxation, for which the impact of replica symmetry breaking is demonstrated. A comparative spectral efficiency analysis reveals significant performance gains compared to linear ZF precoding in the medium to high E-b/N-0 region. The performance vs. complexity tradeoff of the nonlinear precoding scheme is also shortly discussed.