A Federated Meta-Reinforcement Learning Algorithm Based on Gradient Correction

被引:0
|
作者
Qin, Zerui [1 ]
Yue, Sheng [1 ]
机构
[1] Tsinghua Univ, Beijing, Peoples R China
关键词
D O I
10.1145/3674399.3674473
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
To meet the requirement of the decision-making capabilities of IoT devices in dynamic environments, Federated Reinforcement Learning (FRL) has attracted increasing attention in recent years. However, Due to the challenges of heterogeneous environments and sampling limitation in the practical scenarios, it is difficult for clients in different environments to jointly train a personalized policy that can quickly adapt to a new environment. In this paper, we propose a federated meta-reinforcement learning algorithm with gradient correction to extract a meta-policy from heterogeneous environments. Further, we analyze the performance of our proposed algorithm in the experiment section.
引用
收藏
页码:220 / 221
页数:2
相关论文
共 50 条
  • [1] A Meta-Reinforcement Learning-Based Poisoning Attack Framework Against Federated Learning
    Zhou, Wei
    Zhang, Donglai
    Wang, Hongjie
    Li, Jinliang
    Jiang, Mingjian
    IEEE ACCESS, 2025, 13 : 28628 - 28644
  • [2] Meta-Reinforcement Learning Algorithm Based on Reward and Dynamic Inference
    Chen, Jinhao
    Zhang, Chunhong
    Hu, Zheng
    ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, PT III, PAKDD 2024, 2024, 14647 : 223 - 234
  • [3] A Meta-Reinforcement Learning Algorithm for Causal Discovery
    Sauter, Andreas
    Acar, Erman
    Francois-Lavet, Vincent
    CONFERENCE ON CAUSAL LEARNING AND REASONING, VOL 213, 2023, 213 : 602 - 619
  • [4] A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning
    Liu, Bo
    Feng, Xidong
    Ren, Jie
    Mai, Luo
    Zhu, Rui
    Zhang, Haifeng
    Wang, Jun
    Yang, Yaodong
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [5] Hypernetworks in Meta-Reinforcement Learning
    Beck, Jacob
    Jackson, Matthew
    Vuorio, Risto
    Whiteson, Shimon
    CONFERENCE ON ROBOT LEARNING, VOL 205, 2022, 205 : 1478 - 1487
  • [6] Model-based Adversarial Meta-Reinforcement Learning
    Lin, Zichuan
    Thomas, Garrett
    Yang, Guangwen
    Ma, Tengyu
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
  • [7] Model-Based Meta-reinforcement Learning for Hyperparameter Optimization
    Albrechts, Jeroen
    Martin, Hugo M.
    Tavakol, Maryam
    INTELLIGENT DATA ENGINEERING AND AUTOMATED LEARNING - IDEAL 2024, PT I, 2025, 15346 : 27 - 39
  • [8] Prefrontal cortex as a meta-reinforcement learning system
    Jane X. Wang
    Zeb Kurth-Nelson
    Dharshan Kumaran
    Dhruva Tirumala
    Hubert Soyer
    Joel Z. Leibo
    Demis Hassabis
    Matthew Botvinick
    Nature Neuroscience, 2018, 21 : 860 - 868
  • [9] Offline Meta-Reinforcement Learning for Industrial Insertion
    Zhao, Tony Z.
    Luo, Jianlan
    Sushkov, Oleg
    Pevceviciute, Rugile
    Heess, Nicolas
    Scholz, Jon
    Schaal, Stefan
    Levine, Sergey
    2022 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, ICRA 2022, 2022, : 6386 - 6393
  • [10] A Meta-Reinforcement Learning Approach to Process Control
    McClement, Daniel G.
    Lawrence, Nathan P.
    Loewen, Philip D.
    Forbes, Michael G.
    Backstrom, Johan U.
    Gopaluni, R. Bhushan
    IFAC PAPERSONLINE, 2021, 54 (03): : 685 - 692