Distributed learning of energy contracts negotiation strategies with collaborative reinforcement learning

The evolution of electricity markets towards local energy trading models, including peer-to-peer transactions, is bringing by multiple challenges for the involved players. In particular, small consumers, prosumers and generators, with no experience on participating in competitive energy markets, are not prepared for facing such an environment. This paper addresses this problem by proposing a decision support solution for small players negotiations in local transactions. The collaborative reinforcement learning concept is applied to combine different learning processes and reached an enhanced final decision for players actions in bilateral negotiations. The reinforcement learning process is based on the application of the Q-Learning algorithm; and the continuous combination of the different learning results applies and compares several collaborative learning algorithms, namely BEST-Q, Average (AVE)-Q; Particle Swarm Optimization (PSO)-Q, and Weighted Strategy Sharing (WSS)-Q and uses a model to aggregate these results. Results show that the collaborative learning process enables players' to correctly identify the negotiation strategy to apply in each moment, context and against each opponent.