Machine Learning in District Heating System Energy Optimization

This paper introduces a work in progress, where we intend to investigate the application of Reinforcement Learning (RL) and online Supervised Learning (SL) to achieve energy optimization in District-Heating (DH) systems. We believe RL is an ideal approach since this task falls under the control-optimization problem where RL has yielded optimal results in previous work. The magnitude and scale of a DH system complexity incurs the curse of dimensionalities and model, hereby making RL a good choice since it provides a solution for the problem. To assist RL even further with the curse of dimensionalities, we intend to investigate the use of SL to reduce the state space. To achieve this, we shall use historical data to generate a heat load sub-model for each home. We believe using the output of these sub-models as feedback to the RL algorithm could significantly reduce the complexity of the learning task. Also, it could reduce convergence time for the RL algorithm. The desired goal is to achieve a real-time application, which takes operational actions when it receives new direct feedback. However, considering the dynamics of DH system such as large time delay and dissipation in DH network due to various factors, we hope to investigate things such as the appropriate data sampling rate and new parameters / sensors that could improve knowledge about the state of the system, especially on the consumer side of the DH network.