Deep Reinforcement Learning Based Double-layer Optimization Method for Energy Management of Microgrid

被引:0
|
作者
Yu, Qinglei [1 ]
Xu, Wei [2 ]
Lv, Jianhu [3 ]
Wang, Ying [1 ]
Zhang, Kaifeng [1 ]
机构
[1] Southeast Univ, Key Lab Measurement & Control CSE, Minist Educ, Nanjing, Peoples R China
[2] State Grid Jiaxing Power Supply Co, Jiaxing, Zhejiang, Peoples R China
[3] China Elect Power Res Inst, Nanjing, Peoples R China
关键词
microgrid; energy management; deep reinforcement learning; nonlinear programming; OPERATION; MODEL;
D O I
10.1109/AEEES56888.2023.10114319
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Microgrid provides an effective way to integrate renewable energy into power grid. However, the uncertainty of renewable energy and load demand bring great challenges to the energy management of microgrid. Therefore, this paper proposes a double-layer optimization method based on deep reinforcement learning (DRL) to solve this problem. The upper DRL agent takes Soft actor-critic algorithm to fully explore the regulation ability of the energy storage system. The lower nonlinear programming solver optimizes the output of other controllable equipment based on the output of the upper layer, and constantly revises the network parameters of the upper layer according to the optimization results. By combining DRL with traditional nonlinear programming, the convergence speed of the algorithm can be improved and the design difficulty of the DRL reward function can be reduced. Case studies show that the double-layer collaborative optimization method can provide real-time high-quality solutions for energy management of the microgrid only based on the immediate information of the microgrid and can effectively accelerate the convergence speed of the model.
引用
收藏
页码:1016 / 1022
页数:7
相关论文
共 50 条
  • [31] Reinforcement Learning Based Energy Dispatch Strategy and Control Optimization of Microgrid
    Liu, Jin-Hua
    Ke, Zhong-Ming
    Zhou, Wen-Hui
    [J]. Beijing Youdian Daxue Xuebao/Journal of Beijing University of Posts and Telecommunications, 2020, 43 (01): : 28 - 34
  • [32] Research on Operation-Planning Double-Layer Optimization Design Method for Multi-Energy Microgrid Considering Reliability
    Ge, Shaoyun
    Li, Jifeng
    Liu, Hong
    Sun, Hao
    Wang, Yiran
    [J]. APPLIED SCIENCES-BASEL, 2018, 8 (11):
  • [33] Hyperparameter Optimization for Deep Reinforcement Learning in Vehicle Energy Management
    Liessner, Roman
    Schmitt, Jakob
    Dietermann, Ansgar
    Baeker, Bernard
    [J]. PROCEEDINGS OF THE 11TH INTERNATIONAL CONFERENCE ON AGENTS AND ARTIFICIAL INTELLIGENCE (ICAART), VOL 2, 2019, : 134 - 144
  • [34] A two-layer energy management for islanded microgrid based on inverse reinforcement learning and distributed ADMM
    Huang, Lei
    Sun, Wei
    Li, Qiyue
    Mu, Daoming
    Li, Weitao
    [J]. Energy, 2024, 301
  • [35] Optimization of Operation Strategy of Multi-Islanding Microgrid Based on Double-Layer Objective
    Shi, Zheng
    Yan, Lu
    Hu, Yingying
    Wang, Yao
    Qin, Wenping
    Liang, Yan
    Zhao, Haibo
    Jing, Yongming
    Deng, Jiaojiao
    Zhang, Zhi
    [J]. ENERGIES, 2024, 17 (18)
  • [36] An Assessment of Multistage Reward Function Design for Deep Reinforcement Learning-Based Microgrid Energy Management
    Goh, Hui Hwang
    Huang, Yifeng
    Lim, Chee Shen
    Zhang, Dongdong
    Liu, Hui
    Dai, Wei
    Kurniawan, Tonni Agustiono
    Rahman, Saifur
    [J]. IEEE TRANSACTIONS ON SMART GRID, 2022, 13 (06) : 4300 - 4311
  • [37] The Multi-Time-Scale Management Optimization Method for Park Integrated Energy System Based on the Bi-Layer Deep Reinforcement Learning
    Chen, Minghao
    Sun, Yi
    Xie, Zhiyuan
    [J]. Diangong Jishu Xuebao/Transactions of China Electrotechnical Society, 2023, 38 (07): : 1864 - 1881
  • [38] Model-Based Reinforcement Learning Method for Microgrid Optimization Scheduling
    Yao, Jinke
    Xu, Jiachen
    Zhang, Ning
    Guan, Yajuan
    [J]. SUSTAINABILITY, 2023, 15 (12)
  • [39] Energy Management System for an Industrial Microgrid Using Optimization Algorithms-Based Reinforcement Learning Technique
    Upadhyay, Saugat
    Ahmed, Ibrahim
    Mihet-Popa, Lucian
    [J]. ENERGIES, 2024, 17 (16)
  • [40] Double-layer microgrid energy management system for strategic short-term operation scheduling
    Znidarec, Matej
    Sljivac, Damir
    Knezevic, Goran
    Pandzic, Hrvoje
    [J]. INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS, 2024, 157