Adaptive Control of Data Center Cooling using Deep Reinforcement Learning

In this paper, we explore the use of Reinforcement Learning (RL) to improve the control of cooling equipment in Data Centers (DCs). DCs are inherently complex systems, and thus challenging to model from first principles. Machine learning offers a way to address this by instead training a model to capture the thermal dynamics of a DC. In RL, an agent learns to control a system through trial-and-error. However, for systems such as DCs, an interactive trial-and-error approach is not possible, and instead, a high-fidelity model is needed. In this paper, we develop a DC model using Computational Fluid Dynamics (CFD) based on the Lattice Boltzmann Method (LBM) Bhatnagar-Gross-Krook (BGK) algorithm. The model features transient boundary conditions for simulating the DC room, heat-generating servers, and Computer Room Air Handlers (CRAHs) as well as rejection components outside the server room such as heat exchangers, compressors, and dry coolers. This model is used to train an RL agent to control the cooling equipment. Evaluations show that the RL agent can outperform traditional controllers and also can adapt to changes in the environment, such as equipment breaking down.