Virtual power plant containing electric vehicles scheduling strategies based on deep reinforcement learning

Virtual power plants (VPPs), which aggregate customer-side flexibility resources, provide an effective way for customers to participate in the electricity market, and provide a variety of flexible technologies and services to the market. Importantly, VPPs can provide services to electric vehicle (EV) charging stations. In this paper, we constructed a deep reinforcement learning (DRL) based Stackelberg game model for a VPP with EV charging stations. Considering the interests of both sides of the game, soft actor-critic (SAC) algorithm is used for the VPP agent and twin delay deep deterministic policy gradient (TD3) algorithm is used for the EV charging station agent. By alternately training the network parameters of the agents, the strategy and solution at the equilibrium of the game are calculated. Results of cases demonstrate that the VPP agent can learn the strategy of selling electricity to EVs, optimize the scheduling of distributed energy resources (DERs), and bidding strategy for participation in the electricity market. Meanwhile, the EV aggregation agent can learn scheduling strategies for charging and discharging EVs. When the EV aggregator uses a deterministic strategy and the virtual power plant uses a stochastic strategy, energy complementarity is achieved and the overall operating economy is improved.