Joint Resource Scheduling of the Time Slot, Power, and Main Lobe Direction in Directional UAV Ad Hoc Networks: A Multi-Agent Deep Reinforcement Learning Approach

被引:0
|
作者
Liang, Shijie [1 ,2 ]
Zhao, Haitao [2 ]
Zhou, Li [2 ]
Wang, Zhe [2 ]
Cao, Kuo [2 ]
Wang, Junfang [1 ]
机构
[1] China Elect Technol Grp Corp, Res Inst 54, Shijiazhuang 050081, Peoples R China
[2] Natl Univ Def Technol, Coll Elect Sci & Technol, Changsha 410073, Peoples R China
基金
中国国家自然科学基金;
关键词
directional UAV ad hoc network; resource scheduling; multi-agent deep reinforcement learning; attention mechanism; transmission fairness; ALLOCATION; COMMUNICATION; ACCESS;
D O I
10.3390/drones8090478
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
Directional unmanned aerial vehicle (UAV) ad hoc networks (DUANETs) are widely applied due to their high flexibility, strong anti-interference capability, and high transmission rates. However, within directional networks, complex mutual interference persists, necessitating scheduling of the time slot, power, and main lobe direction for all links to improve the transmission performance of DUANETs. To ensure transmission fairness and the total count of transmitted data packets for the DUANET under dynamic data transmission demands, a scheduling algorithm for the time slot, power, and main lobe direction based on multi-agent deep reinforcement learning (MADRL) is proposed. Specifically, modeling is performed with the links as the core, optimizing the time slot, power, and main lobe direction variables for the fairness-weighted count of transmitted data packets. A decentralized partially observable Markov decision process (Dec-POMDP) is constructed for the problem. To process the observation in Dec-POMDP, an attention mechanism-based observation processing method is proposed to extract observation features of UAVs and their neighbors within the main lobe range, enhancing algorithm performance. The proposed Dec-POMDP and MADRL algorithms enable distributed autonomous decision-making for the resource scheduling of time slots, power, and main lobe directions. Finally, the simulation and analysis are primarily focused on the performance of the proposed algorithm and existing algorithms across varying data packet generation rates, different main lobe gains, and varying main lobe widths. The simulation results show that the proposed attention mechanism-based MADRL algorithm enhances the performance of the MADRL algorithm by 22.17%. The algorithm with the main lobe direction scheduling improves performance by 67.06% compared to the algorithm without the main lobe direction scheduling.
引用
收藏
页数:27
相关论文
共 50 条
  • [41] Multi-Agent Deep Reinforcement Learning for Joint Decoupled User Association and Trajectory Design in Full-Duplex Multi-UAV Networks
    Dai, Chen
    Zhu, Kun
    Hossain, Ekram
    IEEE TRANSACTIONS ON MOBILE COMPUTING, 2023, 22 (10) : 6056 - 6070
  • [42] Multi-Agent Deep Reinforcement Learning-Based Resource Allocation for Cognitive Radio Networks
    Mei, Ruru
    Wang, Zhugang
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2025, 74 (03) : 4744 - 4757
  • [43] Multi-Agent Deep Reinforcement Learning for Distributed Resource Management in Wirelessly Powered Communication Networks
    Hwang, Sangwon
    Kim, Hanjin
    Lee, Hoon
    Lee, Inkyu
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2020, 69 (11) : 14055 - 14060
  • [44] A deep multi-agent reinforcement learning approach to solve dynamic job shop scheduling problem
    Liu, Renke
    Piplani, Rajesh
    Toro, Carlos
    COMPUTERS & OPERATIONS RESEARCH, 2023, 159
  • [45] Task offloading and resource allocation for multi-UAV asset edge computing with multi-agent deep reinforcement learning
    Samah A. Zakaryia
    Mohamed Meaad
    Tamer Nabil
    Mohamed K. Hussein
    Computing, 2025, 107 (5)
  • [46] Deep Reinforcement Learning Approach for Joint Trajectory Design in Multi-UAV IoT Networks
    Xu, Shu
    Zhan, Xiangyu
    Li, Chunguo
    Wang, Dongming
    Yang, Luxi
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2022, 71 (03) : 3389 - 3394
  • [47] Joint Topology Construction and Power Adjustment for UAV Networks: A Deep Reinforcement Learning Based Approach
    Xu, Wenjun
    Lei, Huangchun
    Shang, Jin
    CHINA COMMUNICATIONS, 2021, 18 (07) : 265 - 283
  • [48] Joint Topology Construction and Power Adjustment for UAV Networks: A Deep Reinforcement Learning Based Approach
    Wenjun
    Huangchun Lei
    Jin Shang
    中国通信, 2021, 18 (07) : 265 - 283
  • [49] Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning
    Guo, Delin
    Tang, Lan
    Zhang, Xinggan
    Liang, Ying-Chang
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2020, 69 (11) : 13124 - 13138
  • [50] Joint Communication-Motion Planning for UAV Swarm against Jamming with Multi-Agent Deep Reinforcement Learning
    Guo, Zhenxin
    Liu, Yiming
    Wang, Yipeng
    Meng, Yue
    Liu, Baoling
    IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, PIMRC, 2024,