Deep Reinforcement Learning Based Optimal Energy Storage System Operation of Photovoltaic Power Stations With Energy Storage in Power Market

Photovoltaic power stations with energy storage (PV-ES) can not only effectively reduce the real-time variation of the PV’s output, but also be a potential market entity that provides with electric energy and regulation ancillary services. To achieve the above three goals, the ES scheduling strategy should be coordinated with the PV’s outputs. However, at present, most ES scheduling strategies of the PV-ES do not simultaneously coordinate with the reduction of the PV’s real-time output variation and participate in the energy and regulation ancillary service market. On the other hand, the uncertainties, such as the power market prices, or the frequency regulation signals, combining with the ES scheduling strategy, are turning the ES operation problem of PV-ES into a stochastic dynamic non-convex optimal problem. Most of the existing relevant studies use the stochastic scene method or the intelligent algorithm to deal with the non-convex optimization, resulting in the limitation of ES operation and the difficulties of dynamically formulating the operation scheme of ES according to the real-time data. To close the gap, a Deep Q-network based optimal ES operation strategy of PV-ES in power market is proposed. This proposed closed-loop capacity scheduling strategy of ES can not only cope with the non-convex problem, but also autonomously schedule the ES hourly capacity and obtain the maximum of PV-ES profits considering the cost of deviation. © 2022 Power System Technology Press. All rights reserved.