QELAR: A Q-learning-based Energy-Efficient and Lifetime-Aware Routing Protocol for Underwater Sensor Networks

Underwater sensor network (UWSN) has emerged as a promising network technique for various aquatic application in recent years. Due to some constraints in UWSNs, such as high latency, low bandwidth and high energy consumption, it is challenging to build networking protocols for UWSNs. In this paper, we focus on addressing the routing issue in UWSNs. We propose an adaptive, energy-efficient, and lifetime-aware routing protocol based on reinforcement learning, QELAR. Our protocol assumes generic MAC protocols and aims at prolonging the lifetime of networks by making residual energy of sensor nodes more evenly distributed. The residual energy of each node as well as the energy distribution among a group is factored in throughout the routing process to calculate the reward function, which aids in selecting the adequate forwarders for packets. We have performed extensive simulations of the proposed protocol on the Aqua-sim platform, and compared with one existing routing protocol (VBF) in terms of packet delivery rate, energy efficiency, latency and lifetime. The results show that the QELAR protocol yields 20% longer lifetime on average than VBF.