Channel Regularization and Vector Perturbation for Dual-Hop Precoded AF Relaying in Downlink Transmission

In this paper, we investigate the channel regularization and vector perturbation techniques at the source and linear precoding at the relay to enhance the performance of a dual-hop amplify-and-forward (AF) relay downlink system. At first, an optimal regularized channel inversion scheme via regularization parameter is proposed. The joint channel regularization and linear precoder design at the relay is then investigated using the mean-square-error (MSE) criterion. Since dealing directly with such a criterion is quite involved, we propose a simple iterative solution. Our simulation results show that the proposed solution significantly outperforms the conventional channel inversion method. To further improve the performance of the regularization scheme, we further incorporate vector perturbation into channel regularization using a sphere encoder at the source. Numerical results show that the bit-error-rate (BER) performance of the regularized perturbation technique outperforms other well-known precoder designs in dual-hop AF relay downlink systems.