Forced convection heat transfer control for cylinder via closed-loop continuous goal-oriented reinforcement learning

被引:1
|
作者
Liu, Yangwei [1 ,2 ]
Wang, Feitong [1 ,2 ]
Zhao, Shihang [1 ,2 ]
Tang, Yumeng [1 ,2 ]
机构
[1] Beihang Univ, Sch Energy & Power Engn, Beijing 100191, Peoples R China
[2] Beihang Univ, Natl Key Lab Sci & Technol Aeroengine Aerothermody, Beijing 100191, Peoples R China
基金
中国国家自然科学基金;
关键词
FLOW; PERFORMANCE;
D O I
10.1063/5.0239718
中图分类号
O3 [力学];
学科分类号
08 ; 0801 ;
摘要
Forced convection heat transfer control offers considerable engineering value. This study focuses on a two-dimensional rapid temperature control problem in a heat exchange system, where a cylindrical heat source is immersed in a narrow cavity. First, a closed-loop continuous deep reinforcement learning (DRL) framework based on the deep deterministic policy gradient (DDPG) algorithm is developed. This framework swiftly achieves the target temperature with a temperature variance of 0.0116, which is only 5.7% of discrete frameworks. Particle tracking technology is used to analyze the evolution of flow and heat transfer under different control strategies. Due to the broader action space for exploration, continuous algorithms inherently excel in addressing delicate control issues. Furthermore, to address the deficiency that traditional DRL-based active flow control (AFC) frameworks require retraining with each goal changes and cost substantial computational resources to develop strategies for varied goals, the goal information is directly embedded into the agent, and the hindsight experience replay (HER) is employed to improve the training stability and sample efficiency. Then, a closed-loop continuous goal-oriented reinforcement learning (GoRL) framework based on the HER-DDPG algorithm is first proposed to perform real-time rapid temperature transition control and address multiple goals without retraining. Generalization tests show the proposed GoRL framework accomplishes multi-goal tasks with a temperature variance of 0.0121, which is only 5.8% of discrete frameworks, and consumes merely 11% of the computational resources compared with frameworks without goal-oriented capability. The GoRL framework greatly enhances the ability of AFC systems to handle multiple targets and time-varying goals.
引用
收藏
页数:17
相关论文
共 50 条
  • [31] Constrained Reinforcement Learning-Based Closed-Loop Reference Model for Optimal Tracking Control of Unknown Continuous-Time Systems
    Zhang, Haoran
    Zhao, Chunhui
    Ding, Jinliang
    IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2024, 21 (04) : 7312 - 7324
  • [32] Heat and mass transfer analysis of natural convection in a liquid desiccant closed-loop system: The effect of heat source and heat sink temperature
    Xiao, Caiyuan
    Zhang, Guiju
    Hu, PeiSi
    Yu, Yudong
    Mo, YouYu
    Fazilati, MohammadAli
    Toghraie, Davood
    ENERGY REPORTS, 2022, 8 : 1816 - 1828
  • [33] Closed-Loop Continuous Hand Control via Chronic Recording of Regenerative Peripheral Nerve Interfaces
    Vu, Philip P.
    Irwin, Zachary T.
    Bullard, Autumn J.
    Ambani, Shoshana W.
    Sando, Ian C.
    Urbanchek, Melanie G.
    Cederna, Paul S.
    Chestek, Cynthia A.
    IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 2018, 26 (02) : 515 - 526
  • [34] Closed-Loop Feedback Control of a Continuous Pharmaceutical Tablet Manufacturing Process via Wet Granulation
    Ravendra Singh
    Dana Barrasso
    Anwesha Chaudhury
    Maitraye Sen
    Marianthi Ierapetritou
    Rohit Ramachandran
    Journal of Pharmaceutical Innovation, 2014, 9 : 16 - 37
  • [35] Closed-Loop Feedback Control of a Continuous Pharmaceutical Tablet Manufacturing Process via Wet Granulation
    Singh, Ravendra
    Barrasso, Dana
    Chaudhury, Anwesha
    Sen, Maitraye
    Ierapetritou, Marianthi
    Ramachandran, Rohit
    JOURNAL OF PHARMACEUTICAL INNOVATION, 2014, 9 (01) : 16 - 37
  • [36] Free and forced convection heat transfer in the thermal entry region for laminar flow inside a circular cylinder horizontally oriented
    Mohammed, Hussein A.
    Salman, Yasin K.
    ENERGY CONVERSION AND MANAGEMENT, 2007, 48 (07) : 2185 - 2195
  • [37] A continuous-time closed-loop identification method based on iterative learning control concepts
    Sakai, Fumitoshi
    Sugie, Toshiharu
    PROCEEDINGS OF THE 46TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-14, 2007, : 3245 - +
  • [38] Numerical heat transfer simulations of a vertical batch furnace under closed-loop temperature control
    Wilson, GJ
    McHugh, PR
    Weaver, RA
    PROCEEDINGS OF THE SECOND INTERNATIONAL SYMPOSIUM ON PROCESS CONTROL, DIAGNOSTICS, AND MODELING IN SEMICONDUCTOR MANUFACTURING, 1997, 97 (09): : 110 - 117
  • [39] Closed-loop individual cylinder air-fuel ratio control via UEGO signal spectral analysis
    Cavina, Nicolo
    Corti, Enrico
    Moro, Davide
    CONTROL ENGINEERING PRACTICE, 2010, 18 (11) : 1295 - 1306
  • [40] Improving ceramic additive manufacturing via machine learning-enabled closed-loop control
    Zhang, Zhaolong
    Yang, Zhaotong
    Sisson, Richard D.
    Liang, Jianyu
    INTERNATIONAL JOURNAL OF APPLIED CERAMIC TECHNOLOGY, 2022, 19 (02) : 957 - 967