Optimal Reliability in Energy Harvesting Industrial Wireless Sensor Networks

For industrial wireless sensor networks, it is essential to reliably sense and deliver the environmental data on time to avoid system malfunction. While energy harvesting is a promising technique to extend the lifetime of sensor nodes, it also brings new challenges for system reliability due to the stochastic nature of the harvested energy. In this paper, we investigate the optimal energy management policy to minimize the weighted packet loss rate under the delay constraint, where the packet loss rate considers the lost packets, both during the sensing and delivering processes. We show that the above-mentioned energy management problem can be modeled as an infinite horizon average reward constraint Markov decision problem. In order to address the well-known curse of dimensionality problem and facilitate distributed implementation, we use the linear value approximation technique. Moreover, we apply stochastic online learning with a post-decision state to deal with the lack of the knowledge of the underlying stochastic processes. A distributed energy allocation algorithm with a water-filling structure and a scheduling algorithm by an auction mechanism are obtained. Experimental results show that the proposed algorithm achieves nearly the same performance as the optimal offline value iteration algorithm while requiring much less computation complexity and signaling overhead, and outperforms various existing baseline algorithms.