Rolling horizon wind-thermal unit commitment optimization based on deep reinforcement learning

The growing penetration of renewable energy has brought significant challenges for modern power system operation. Academic research and industrial practice show that adjusting unit commitment (UC) scheduling periodically according to new forecasts of renewable power provides a promising way to improve system stability and economy; however, this greatly increases the computational burden for solution methods. In this paper, a deep reinforcement learning (DRL) method is proposed to obtain timely and reliable solutions for rolling-horizon UC (RHUC). First, based on historical data and day-ahead point forecasting, a data-driven method is designed to construct typical wind power scenarios that are regarded as components of the state space of DRL. Second, a rolling mechanism is proposed to dynamically update the state space based on real-time wind power data. Third, unlike existing reinforcement learning-based UC solution methods that segment the continuous outputs of generators as discrete variables, all the variables in RHUC are regarded as continuous. Additionally, a series of updating regulations are defined to ensure that the model is realistic. Thus, a DRL algorithm, the twin delayed deep deterministic policy gradient (TD3), can be utilized to effectively solve the problem. Finally, several case studies are conducted based on different test systems to demonstrate the efficiency of the proposed method. According to the experimental results, the proposed algorithm can obtain high-quality solutions in a considerably shorter time than traditional methods, which leads to a reduction of at least 1.1% in the power system operation cost.