Enhancing the Physical Layer Security of Two-Way Relay Systems With RIS and Beamforming

This paper studies the possible physical layer security (PLS) enhancement in a wiretapped two-way relay system from utilizing the reconfigurable intelligent surface (RIS) and beamforming techniques. We first jointly apply the RIS and beamforming (RIS-B) to develop an advanced secure transmission scheme for the system, and conduct related theoretical modeling for the system secrecy sum rate (SSR) under the RIS-B scheme. We then apply the theories of alternating optimization, successive convex approximation, augmented Lagrange method and the quasi-Newton method to devise both the optimal and low-complexity sub-optimal frameworks to identify the optimal settings of RIS phase shifts and beamforming matrix of the RIS-B scheme for SSR maximization. We further investigate a special case when solely the RIS technique is applied for SSR performance enhancement. With the help of Charnes-Cooper transformation, semidefinite programming and Gaussian randomization, we devise an optimization framework to identify the optimal settings of RIS phase shifts for SSR maximization. Finally, extensive numerical results are provided to illustrate the efficiency of the RIS and RIS-B schemes in PLS enhancement in comparison with the conventional channel-capacity-based secure transmission.