TDMA policy to optimize resource utilization in Wireless Sensor Networks using reinforcement learning for ambient environment

Data packet reaches from the end node to sink in a multihop fashion in the internet of things (IoTs) and sensor networks. Usually, a head node (among neighboring or special purpose nodes) can collect data packets from the nodes and forward them further to sink or other head nodes. In Time-division multiple access (TDMA) driven scheduling, nodes often own slots in a time frame and are scheduled for data forwarding in the allotted time slot (owner node) in each time frame. A time frame in which the owner node does not have data to forward goes into sleep mode. Though the supposed owner node is in sleep mode, the corresponding head node is active throughout the time frame. This active period of a head node can cause an increase in energy consumption. Besides, because the head node in an active state does not receive a data packet, it is causing significantly to the throughput, ultimately leading to low channel utilization. We propose the Markov design policy (MDP) for such head nodes to reduce the number of time slots wasted in the time frame in our work. The proposal is the first such kind of MDP-based modeling for node scheduling in TDMA. The simulation results show that the proposed method outperforms existing adaptive scheduling algorithms for channel utilization, end-to-end delay, system utilization, and balance factor.