Forced convection heat transfer control for cylinder via closed-loop continuous goal-oriented reinforcement learning

被引:1
|
作者
Liu, Yangwei [1 ,2 ]
Wang, Feitong [1 ,2 ]
Zhao, Shihang [1 ,2 ]
Tang, Yumeng [1 ,2 ]
机构
[1] Beihang Univ, Sch Energy & Power Engn, Beijing 100191, Peoples R China
[2] Beihang Univ, Natl Key Lab Sci & Technol Aeroengine Aerothermody, Beijing 100191, Peoples R China
基金
中国国家自然科学基金;
关键词
FLOW; PERFORMANCE;
D O I
10.1063/5.0239718
中图分类号
O3 [力学];
学科分类号
08 ; 0801 ;
摘要
Forced convection heat transfer control offers considerable engineering value. This study focuses on a two-dimensional rapid temperature control problem in a heat exchange system, where a cylindrical heat source is immersed in a narrow cavity. First, a closed-loop continuous deep reinforcement learning (DRL) framework based on the deep deterministic policy gradient (DDPG) algorithm is developed. This framework swiftly achieves the target temperature with a temperature variance of 0.0116, which is only 5.7% of discrete frameworks. Particle tracking technology is used to analyze the evolution of flow and heat transfer under different control strategies. Due to the broader action space for exploration, continuous algorithms inherently excel in addressing delicate control issues. Furthermore, to address the deficiency that traditional DRL-based active flow control (AFC) frameworks require retraining with each goal changes and cost substantial computational resources to develop strategies for varied goals, the goal information is directly embedded into the agent, and the hindsight experience replay (HER) is employed to improve the training stability and sample efficiency. Then, a closed-loop continuous goal-oriented reinforcement learning (GoRL) framework based on the HER-DDPG algorithm is first proposed to perform real-time rapid temperature transition control and address multiple goals without retraining. Generalization tests show the proposed GoRL framework accomplishes multi-goal tasks with a temperature variance of 0.0121, which is only 5.8% of discrete frameworks, and consumes merely 11% of the computational resources compared with frameworks without goal-oriented capability. The GoRL framework greatly enhances the ability of AFC systems to handle multiple targets and time-varying goals.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] Lagrangian analysis on mechanism of forced convection heat transfer control for cylinder via deep reinforcement learning
    Liu, Yangwei
    Wang, Feitong
    Zhao, Shihang
    Hou, Jiexuan
    PHYSICS OF FLUIDS, 2025, 37 (01)
  • [2] Closed-Loop Control of Direct Ink Writing via Reinforcement Learning
    Piovarci, Michal
    Foshey, Michael
    Xu, Jie
    Erps, Timmothy
    Babaei, Vahid
    Didyk, Piotr
    Rusinkiewicz, Szymon
    Matusik, Wojciech
    Bickel, Bernd
    ACM TRANSACTIONS ON GRAPHICS, 2022, 41 (04):
  • [3] A goal-oriented reinforcement learning for optimal drug dosage control
    Zhang, Qian
    Li, Tianhao
    Li, Dengfeng
    Lu, Wei
    ANNALS OF OPERATIONS RESEARCH, 2024, 338 (2-3) : 1403 - 1423
  • [4] Closed-loop control of a noisy qubit with reinforcement learning
    Ding, Yongcheng
    Chen, Xi
    Magdalena-Benedito, Rafael
    Martin-Guerrero, Jose D.
    MACHINE LEARNING-SCIENCE AND TECHNOLOGY, 2023, 4 (02):
  • [5] Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space
    Cimurs, Reinis
    Lee, Jin Han
    Suh, Il Hong
    ELECTRONICS, 2020, 9 (03)
  • [6] Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer
    Wang, Yi-Zhe
    Li, Yu-Bai
    Aubry, Nadine
    Hua, Yue
    Zhou, Zhi-Fu
    Chen, Zhi-Hua
    Wu, Wei-Tao
    PHYSICS OF FLUIDS, 2023, 35 (07)
  • [7] Control policy transfer of deep reinforcement learning based intelligent forced heat convection control
    Wang, Yi-Zhe
    Peng, Jiang-Zhou
    Aubry, Nadine
    Li, Yu-Bai
    Chen, Zhi-Hua
    Wu, Wei-Tao
    INTERNATIONAL JOURNAL OF THERMAL SCIENCES, 2024, 195
  • [8] Path planning via reinforcement learning with closed-loop motion control and field tests
    Feher, Arpad
    Domina, Adam
    Bardos, Adam
    Aradi, Szilard
    Becsi, Tamas
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2025, 142
  • [9] Deep Reinforcement Learning for Closed-Loop Blood Glucose Control
    Fox, Ian
    Lee, Joyce
    Pop-Busui, Rodica
    Wiens, Jenna
    MACHINE LEARNING FOR HEALTHCARE CONFERENCE, VOL 126, 2020, 126 : 508 - 535
  • [10] Closed-Loop Control of Fluid Resuscitation Using Reinforcement Learning
    Estiri, Elham
    Mirinejad, Hossein
    IEEE ACCESS, 2023, 11 : 140569 - 140581