A novel nonlinear precoding algorithm for the downlink of multiple antenna multi-user systems

By pre-equalizing inter-layer interference at the transmitter, Tomlinson-Harashima precoding (THP) algorithm provides a solution for the downlink of multiple antenna multi-user systems, in which the decentralized structure of the receivers makes the receiver-processing algorithms impossible. However, for the zero-forcing (ZF) THP algorithm developed in the literature there are significant performance differences between specific mobile stations. In this paper, a novel version of the THIP algorithm is proposed. It greatly improves the worst mobile's performance and ensures balanced performance of all the mobiles. For the new THP algorithm, better performance can be obtained by suitably ordering the rows of the channel matrix. We show that the "best-first" ordering method achieves optimal order for BER performance in 2x2 systems and achieves near optimal order in systems of larger dimensions. Simulation is used to show the advantages of the new THP algorithm and the "best-first" ordering method.