On Two-Way Relay Channel with an Active Malicious Relay

Two-way relay channel is a fundamental building block of cooperative wireless networks, where two source nodes exchange messages via relays. However, most works in the literature assume that all relays are fully cooperative and neglect the effect of possibly misbehaving. This paper investigates the security issues in a two-way relay channel where one of the relay nodes is compromised to malicious and actively attacking the communications. As for the cooperative relay in this channel, it adopts the celebrated analog network coding (ANC). We derive the optimal strategies of both relays by modeling the problem as a mutual information game and identifying the Nash equilibrium (NE) of the game. The cooperative and malicious relays are two players in the game and the achievable sum rates are adopted as the utility for the cooperative relay. We consider the scenario that relays can change transmission actions symbol by symbol in a codeword length and prove that the pure strategy NE exists which suggest the cooperative relay to always vary the amplifying coefficients of ANC symbol by symbol even for a fixed channel. The non-convex utility function is reformulated via proper linearization to obtain the closed-form solution of the optimal strategy. Furthermore, we also show that the attack strategies proposed by previous work, where malicious relay should always attack with its received signal when the channels are fixed, are suboptimal.