DGTRL: Deep graph transfer reinforcement learning method based on fusion of knowledge and data

被引:1
|
作者
Chen, Genxin [1 ]
Qi, Jin [2 ]
Gao, Yu [1 ]
Zhu, Xingjian [1 ]
Dong, Zhenjiang [3 ]
Sun, Yanfei [2 ]
机构
[1] Nanjing Univ Posts & Telecommun, Sch Automat, Nanjing, Jiangsu, Peoples R China
[2] Nanjing Univ Posts & Telecommun, Sch Internet Things, Nanjing, Jiangsu, Peoples R China
[3] Nanjing Univ Posts & Telecommun, Sch Comp Sci, Nanjing, Jiangsu, Peoples R China
基金
中国国家自然科学基金;
关键词
Knowledge reuse; Deep reinforcement learning; Graph matching; Transfer learning; Adaptive policy transfer; Fusion; NETWORK;
D O I
10.1016/j.ins.2023.120019
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Deep reinforcement learning has shown promising application effects in many fields. However, issues such as low sample efficiency and weak knowledge transfer and generalization capabilities constrain its in-depth development. This paper proposes a deep graph transfer reinforcement learning method based on knowledge and data fusion. Through knowledge transfer between tasks and data training in exploration, efficient learning driven by knowledge and data fusion is realized. First, we map the trajectories of agents in the source and target domains into graph structures to represent knowledge and environmental models and use a graph matching method to match the knowledge adaptation domain based on similarity. Second, real-time transfer of source domain knowledge is carried out in a similar mapping domain, and policy mapping, verification, and adaptive transfer mechanisms based on knowledge reuse frequency are designed to effectively improve the transfer effect. Finally, we evaluate and validate the proposed algorithm, mechanism, and their application effectiveness in diverse and multitask knowledge transfer scenarios using three indicators in environments such as maze. The results show that the method improves the target task time indicators by 84.7% and 70% under the conditions of no verification exploration and a standard transfer setting, respectively, achieving significant knowledge reuse effects.
引用
收藏
页数:21
相关论文
共 50 条
  • [1] Knowledge Reasoning Method Based on Deep Transfer Reinforcement Learning: DTRLpath
    Lin, Shiming
    Ye, Ling
    Zhuang, Yijie
    Lu, Lingyun
    Zheng, Shaoqiu
    Huang, Chenxi
    Kwee, Ng Yin
    [J]. CMC-COMPUTERS MATERIALS & CONTINUA, 2024, 80 (01): : 299 - 317
  • [2] Dynamic knowledge graph reasoning based on deep reinforcement learning
    Liu, Hao
    Zhou, Shuwang
    Chen, Changfang
    Gao, Tianlei
    Xu, Jiyong
    Shu, Minglei
    [J]. KNOWLEDGE-BASED SYSTEMS, 2022, 241
  • [3] A Dynamic Financial Knowledge Graph Based on Reinforcement Learning and Transfer Learning
    Miao, Rui
    Zhang, Xia
    Yan, Hongfei
    Chen, Chong
    [J]. 2019 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2019, : 5370 - 5378
  • [4] Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning
    Wang, Heng
    Li, Shuangyin
    Pan, Rong
    Mao, Mingzhi
    [J]. 2019 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING AND THE 9TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING (EMNLP-IJCNLP 2019): PROCEEDINGS OF THE CONFERENCE, 2019, : 2623 - 2631
  • [5] TMAPath: A Knowledge Inference Method Based on Transfer Learning and Multi-agent Deep Reinforcement Learning
    He, Dairao
    Hu, Jingfeng
    [J]. Journal of Network Intelligence, 2022, 7 (03): : 574 - 591
  • [6] A Deep Reinforcement Learning Method For Multimodal Data Fusion in Action Recognition
    Guo, Jiale
    Liu, Qiang
    Chen, Enqing
    [J]. IEEE SIGNAL PROCESSING LETTERS, 2022, 29 : 120 - 124
  • [7] Towards Knowledge Transfer in Deep Reinforcement Learning
    Glatt, Ruben
    da Silva, Felipe Leno
    Reali Costa, Anna Helena
    [J]. PROCEEDINGS OF 2016 5TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS 2016), 2016, : 91 - 96
  • [8] Improving Deep Reinforcement Learning with Knowledge Transfer
    Glatt, Ruben
    Reali Costa, Anna Helena
    [J]. THIRTY-FIRST AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2017, : 5036 - 5037
  • [9] REPAINT: Knowledge Transfer in Deep Reinforcement Learning
    Tao, Yunzhe
    Genc, Sahika
    Chung, Jonathan
    Sun, Tao
    Mallya, Sunil
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139 : 7145 - 7155
  • [10] Knowledge Transfer using Model-Based Deep Reinforcement Learning
    Boloka, Tlou
    Makondo, Ndivhuwo
    Rosman, Benjamin
    [J]. 2021 SOUTHERN AFRICAN UNIVERSITIES POWER ENGINEERING CONFERENCE/ROBOTICS AND MECHATRONICS/PATTERN RECOGNITION ASSOCIATION OF SOUTH AFRICA (SAUPEC/ROBMECH/PRASA), 2021,