Demand Response of a Heterogeneous Cluster of Electric Water Heaters Using Batch Reinforcement Learning

A demand response aggregator, that manages a large cluster of heterogeneous flexibility carriers, faces a complex optimal control problem. Moreover, in most applications of demand response an exact description of the system dynamics and constraints is unavailable, and information comes mostly from observations of system trajectories. This paper presents a model-free approach for controlling a cluster of domestic electric water heaters. The objective is to schedule the cluster at minimum electricity cost by using the thermal storage of the water tanks. The control scheme applies a model-free batch reinforcement learning (batch RL) algorithm in combination with a market-based heuristic. The considered batch RL technique is tested in a stochastic setting, without prior information or model of the system dynamics of the cluster. The simulation results show that the batch RL technique is able to reduce the daily electricity cost within a reasonable learning period of 40-45 days, compared to a hysteresis controller.