Deep Reinforcement Learning for Online Scheduling of Photovoltaic Systems with Battery Energy Storage Systems

被引:0
|
作者
Li, Yaze [1 ]
Wu, Jingxian [1 ]
Pan, Yanjun [2 ]
机构
[1] University of Arkansas, Department of Electrical Engineering, Fayetteville,AR,72701, United States
[2] University of Arkansas, Department of Computer Science and Computer Engineering, Fayetteville,AR,72701, United States
来源
Intelligent and Converged Networks | 2024年 / 5卷 / 01期
基金
美国国家科学基金会;
关键词
Deep learning - Digital storage - E-learning - Learning algorithms - Learning systems - Online systems - Reinforcement learning - Scheduling algorithms - Secondary batteries - Stochastic systems;
D O I
10.23919/ICN.2024.0003
中图分类号
学科分类号
摘要
A new online scheduling algorithm is proposed for photovoltaic (PV) systems with battery-assisted energy storage systems (BESS). The stochastic nature of renewable energy sources necessitates the employment of BESS to balance energy supplies and demands under uncertain weather conditions. The proposed online scheduling algorithm aims at minimizing the overall energy cost by performing actions such as load shifting and peak shaving through carefully scheduled BESS charging/discharging activities. The scheduling algorithm is developed by using deep deterministic policy gradient (DDPG), a deep reinforcement learning (DRL) algorithm that can deal with continuous state and action spaces. One of the main contributions of this work is a new DDPG reward function, which is designed based on the unique behaviors of energy systems. The new reward function can guide the scheduler to learn the appropriate behaviors of load shifting and peak shaving through a balanced process of exploration and exploitation. The new scheduling algorithm is tested through case studies using real world data, and the results indicate that it outperforms existing algorithms such as Deep Q-learning. The online algorithm can efficiently learn the behaviors of optimum non-casual off-line algorithms. © 2020 Tsinghua University Press.
引用
收藏
页码:28 / 41
相关论文
共 50 条
  • [1] Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets
    Liu, Yang
    Lu, Qiuyu
    Yu, Zhenfan
    Chen, Yue
    Yang, Yinguo
    [J]. Energies, 2024, 17 (21)
  • [2] Optimal Scheduling of Battery Energy Storage Systems Using a Reinforcement Learning-based Approach
    Selim, Alaa
    Mo, Huadong
    Pota, Hemanshu
    Dong, Daoyi
    [J]. IFAC PAPERSONLINE, 2023, 56 (02): : 11741 - 11747
  • [3] Energy Scheduling for Multi-Energy Systems via Deep Reinforcement Learning
    Wang, Zixin
    Zhu, Shanying
    Ding, Tao
    Yang, Bo
    [J]. 2020 IEEE POWER & ENERGY SOCIETY GENERAL MEETING (PESGM), 2020,
  • [4] Physics-Shielded Multi-Agent Deep Reinforcement Learning for Safe Active Voltage Control With Photovoltaic/Battery Energy Storage Systems
    Chen, Pengcheng
    Liu, Shichao
    Wang, Xiaozhe
    Kamwa, Innocent
    [J]. IEEE TRANSACTIONS ON SMART GRID, 2023, 14 (04) : 2656 - 2667
  • [5] Interpretable Deep Reinforcement Learning for Optimizing Heterogeneous Energy Storage Systems
    Xiong, Luolin
    Tang, Yang
    Liu, Chensheng
    Mao, Shuai
    Meng, Ke
    Dong, Zhaoyang
    Qian, Feng
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS, 2024, 71 (02) : 910 - 921
  • [6] Minimizing Energy Cost in PV Battery Storage Systems Using Reinforcement Learning
    Haertel, Florus
    Bocklisch, Thilo
    [J]. IEEE ACCESS, 2023, 11 : 39855 - 39865
  • [7] Deep reinforcement learning based solution for sustainable energy management in photovoltaic systems
    Alamro, Hayam
    Alqahtani, Hamed
    Alotaibi, Faiz Abdullah
    Othman, Kamal M.
    Assiri, Mohammed
    Alneil, Amani A.
    Narasimha Prasad, L.V.
    [J]. Optik, 2023, 295
  • [8] Deep reinforcement learning-based optimal scheduling of integrated energy systems for electricity, heat, and hydrogen storage
    Liang, Tao
    Zhang, Xiaochan
    Tan, Jianxin
    Jing, Yanwei
    Liangnian, Lv
    [J]. ELECTRIC POWER SYSTEMS RESEARCH, 2024, 233
  • [9] Deep Reinforcement Learning for Hybrid Energy Storage Systems: Balancing Lead and Hydrogen Storage
    Desportes, Louis
    Fijalkow, Inbar
    Andry, Pierre
    [J]. ENERGIES, 2021, 14 (15)
  • [10] Battery Energy Storage Systems Modeling for Online Applications
    Misyris, George S.
    Tengner, Tomas
    Marinopoulos, Antonis G.
    Doukas, Dimitrios I.
    Labridis, Dimitris P.
    [J]. 2017 IEEE MANCHESTER POWERTECH, 2017,