Peer-to-Peer Energy Transaction Decision of Prosumers Based on Reinforcement Learning

Reinforcement learning is an artificial intelligence method to maximize the payback of intelligent agents through learning strategies in the process of interaction with the environment. Without the optimal calculation and the full knowledge of the market mechanism, this method is very suitable for prosumers dealing with the small-scale energy transaction behavior of users. Firstly, a peer-to-peer energy transaction model including transaction subjects, price and physical constraints is established in this paper. Secondly, the energy transaction problem is equivalent to a Markov decision process and each learning element is modeled. Then, based on the Q-learning reinforcement learning algorithm, the problem of energy storage action and transaction strategy selection in Markov decision process is discretized uniformly, and then analyzed and solved. Finally, the case of energy transaction including multi-type prosumers and consumers is used to verify the rationality and feasibility of the reinforcement learning method in solving the peer-to-peer power transaction problem of small-scale prosumers. © 2021 Automation of Electric Power Systems Press.