Reinforcement Learning for Control of Building HVAC Systems

We propose a reinforcement learning-based (RL) controller for energy efficient climate control of commercial buildings. Model-based control techniques like model predictive control (MPC) for this problem are challenging to implement as they need simple yet accurate models, which are hard to obtain due to the complexity in hygrothermal dynamics of a building and its HVAC system. RL is an attractive alternative to MPC since once the policy is learned, computing the control in real time involves solving a simple low dimensional optimization problem that does not involve a model of building physics. However, training an RL controller is computationally expensive, and there are many design choices that affect performance. We compare in simulations the proposed RL controller, an MPC controller, and a baseline rule-based controller that is widely used in practice. Both the RL and MPC controllers are able to maintain temperature and humidity constraints, and they both reduce energy use significantly compared to the baseline, though the savings by RL is smaller than that by MPC.