UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

被引:13
|
作者
Gong, Zihao [1 ]
Xu, Yang [2 ,3 ]
Luo, Delin [1 ]
机构
[1] Xiamen Univ, Sch Aerosp Engn, Xiamen 361102, Peoples R China
[2] Northwestern Polytech Univ, Sch Civil Aviat, Xian 710072, Peoples R China
[3] Yangtze River Delta Res Inst NPU, Taicang 215400, Peoples R China
基金
中国国家自然科学基金;
关键词
UAV cooperative air combat; MARL; maneuvering decision-making; VDN algorithm; TREE;
D O I
10.1142/S2301385023410029
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Focusing on the problem of multi-UAV cooperative air combat decision-making, a multi-UAV cooperative maneuvering decision-making approach is proposed based on multi-agent deep reinforcement learning (MARL) theory. First, the multi-UAV cooperative short-range air combat environment is established. Then, by combining the value-decomposition networks (VDNs) deep reinforcement learning theory with the embedded expert collaborative air combat experience reward function, an air combat cooperative strategy framework is proposed based on the networked decentralized partially observable Markov decision process (NDec-POMDP). The air combat maneuvering strategy is then optimized to improve the cooperative degree between UAVs in cooperative combat scenarios. Finally, multi-UAV cooperative air combat simulations are carried out and the results show the feasibility and effectiveness of the proposed cooperative air combat decision-making framework and method.
引用
收藏
页码:273 / 286
页数:14
相关论文
共 50 条
  • [31] Learning Cooperative Intrinsic Motivation in Multi-Agent Reinforcement Learning
    Hong, Seung-Jin
    Lee, Sang-Kwang
    [J]. 12TH INTERNATIONAL CONFERENCE ON ICT CONVERGENCE (ICTC 2021): BEYOND THE PANDEMIC ERA WITH ICT CONVERGENCE INNOVATION, 2021, : 1697 - 1699
  • [32] Cooperative Learning of Multi-Agent Systems Via Reinforcement Learning
    Wang, Xin
    Zhao, Chen
    Huang, Tingwen
    Chakrabarti, Prasun
    Kurths, Juergen
    [J]. IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, 2023, 9 : 13 - 23
  • [33] Cooperative Multi-Agent Reinforcement Learning With Approximate Model Learning
    Park, Young Joon
    Lee, Young Jae
    Kim, Seoung Bum
    [J]. IEEE ACCESS, 2020, 8 : 125389 - 125400
  • [34] Cooperative Multi-Agent Reinforcement Learning with Hypergraph Convolution
    Bai, Yunpeng
    Gong, Chen
    Zhang, Bin
    Fan, Guoliang
    Hou, Xinwen
    Lu, Yu
    [J]. 2022 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2022,
  • [35] A review of cooperative multi-agent deep reinforcement learning
    Oroojlooy, Afshin
    Hajinezhad, Davood
    [J]. APPLIED INTELLIGENCE, 2023, 53 (11) : 13677 - 13722
  • [36] Levels of Realism for Cooperative Multi-agent Reinforcement Learning
    Cunningham, Bryan
    Cao, Yong
    [J]. ADVANCES IN SWARM INTELLIGENCE, ICSI 2012, PT I, 2012, 7331 : 573 - 582
  • [37] The Cooperative Reinforcement Learning in a Multi-Agent Design System
    Liu, Hong
    Wang, Jihua
    [J]. PROCEEDINGS OF THE 2013 IEEE 17TH INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED COOPERATIVE WORK IN DESIGN (CSCWD), 2013, : 139 - 144
  • [38] Cooperative Multi-Agent Reinforcement Learning in Express System
    Li, Yexin
    Zheng, Yu
    Yang, Qiang
    [J]. CIKM '20: PROCEEDINGS OF THE 29TH ACM INTERNATIONAL CONFERENCE ON INFORMATION & KNOWLEDGE MANAGEMENT, 2020, : 805 - 814
  • [39] Cooperative Multi-agent Reinforcement Learning for Inventory Management
    Khirwar, Madhav
    Gurumoorthy, Karthik S.
    Jain, Ankit Ajit
    Manchenahally, Shantala
    [J]. MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES: APPLIED DATA SCIENCE AND DEMO TRACK, ECML PKDD 2023, PT VI, 2023, 14174 : 619 - 634
  • [40] A review of cooperative multi-agent deep reinforcement learning
    Afshin Oroojlooy
    Davood Hajinezhad
    [J]. Applied Intelligence, 2023, 53 : 13677 - 13722