Secure Beamforming for Full-duplex MIMO Two-way Communication via Untrusted Relaying

Full-duplex (FD) technique is very promising to support enormous data for 5G networks since it can greatly improve the spectrum efficiency. However, the secrecy performance of FD communication systems still remains largely unknown. As an initial step towards this end, this paper studies the physical layer security-based beamforming design in a FD two-way amplify-and-forward relay system, where two source nodes exchange messages with the assistance of an untrusted relay. The untrusted relay is willing to forward the messages and may also work as an eavesdropper to intercept the confidential messages. Our objective is to optimize the source and relay beamformers for maximizing the secrecy sum rate of the two-way communications with imperfect self-interference cancellation. For a specific system configuration, we first derive the expression of secrecy sum rate. By utilizing signal alignment technique and gradient method, we then propose two iterative algorithms to find the optimal source and relay beamforming matrices, respectively. Simulation results demonstrate that when the number of antennas at the relay is appropriate and the interference-to-noise ratio is in a low region the proposed FD scheme significantly improves the secrecy sum rate over conventional schemes.