Multi-agent reinforcement learning for strategic bidding in power markets

In the agent-based simulation discussed in this paper, we study the dynamics of the power market, when suppliers act following a Q-learning based bidding strategy. Power suppliers aim to satisfy two objectives: the maximization of their profit and their utilization rate. To meet with success their goals, they need to acquire a complex behavior by learning through a continuous exploiting and exploring process. Reinforcement learning theory provides a formal framework, along with a family of learning methods. In this paper we use Q-learning algorithm, perhaps the most popular among temporal difference methods. Q-learning offers suppliers the ability to evaluate their actions and to retain the most profitable of them. A five bus power system is used for our case studies; our experiments are contacted with three supplier-agents in all cases but the last one where nine agents participate. The Locational Marginal Pricing (LMP) system serves as the market clearing mechanism.