Joint Resource Scheduling of the Time Slot, Power, and Main Lobe Direction in Directional UAV Ad Hoc Networks: A Multi-Agent Deep Reinforcement Learning Approach

被引:0
|
作者
Liang, Shijie [1 ,2 ]
Zhao, Haitao [2 ]
Zhou, Li [2 ]
Wang, Zhe [2 ]
Cao, Kuo [2 ]
Wang, Junfang [1 ]
机构
[1] China Elect Technol Grp Corp, Res Inst 54, Shijiazhuang 050081, Peoples R China
[2] Natl Univ Def Technol, Coll Elect Sci & Technol, Changsha 410073, Peoples R China
基金
中国国家自然科学基金;
关键词
directional UAV ad hoc network; resource scheduling; multi-agent deep reinforcement learning; attention mechanism; transmission fairness; ALLOCATION; COMMUNICATION; ACCESS;
D O I
10.3390/drones8090478
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
Directional unmanned aerial vehicle (UAV) ad hoc networks (DUANETs) are widely applied due to their high flexibility, strong anti-interference capability, and high transmission rates. However, within directional networks, complex mutual interference persists, necessitating scheduling of the time slot, power, and main lobe direction for all links to improve the transmission performance of DUANETs. To ensure transmission fairness and the total count of transmitted data packets for the DUANET under dynamic data transmission demands, a scheduling algorithm for the time slot, power, and main lobe direction based on multi-agent deep reinforcement learning (MADRL) is proposed. Specifically, modeling is performed with the links as the core, optimizing the time slot, power, and main lobe direction variables for the fairness-weighted count of transmitted data packets. A decentralized partially observable Markov decision process (Dec-POMDP) is constructed for the problem. To process the observation in Dec-POMDP, an attention mechanism-based observation processing method is proposed to extract observation features of UAVs and their neighbors within the main lobe range, enhancing algorithm performance. The proposed Dec-POMDP and MADRL algorithms enable distributed autonomous decision-making for the resource scheduling of time slots, power, and main lobe directions. Finally, the simulation and analysis are primarily focused on the performance of the proposed algorithm and existing algorithms across varying data packet generation rates, different main lobe gains, and varying main lobe widths. The simulation results show that the proposed attention mechanism-based MADRL algorithm enhances the performance of the MADRL algorithm by 22.17%. The algorithm with the main lobe direction scheduling improves performance by 67.06% compared to the algorithm without the main lobe direction scheduling.
引用
收藏
页数:27
相关论文
共 50 条
  • [1] Multi-Agent Deep Reinforcement Learning for Cross-Layer Scheduling in Mobile Ad-Hoc Networks
    Xinxing Zheng
    Yu Zhao
    Joohyun Lee
    Wei Chen
    ChinaCommunications, 2023, 20 (08) : 78 - 88
  • [2] Multi-agent deep reinforcement learning for cross-layer scheduling in mobile ad-hoc networks
    Zheng, Xinxing
    Zhao, Yu
    Lee, Joohyun
    Chen, Wei
    CHINA COMMUNICATIONS, 2023, 20 (08) : 78 - 88
  • [3] Intelligent Vehicle Computation Offloading in Vehicular Ad Hoc Networks: A Multi-Agent LSTM Approach with Deep Reinforcement Learning
    Sun, Dingmi
    Chen, Yimin
    Li, Hao
    MATHEMATICS, 2024, 12 (03)
  • [4] Multi-Agent Deep Reinforcement Learning for Trajectory Design and Power Allocation in Multi-UAV Networks
    Zhao, Nan
    Liu, Zehua
    Cheng, Yiqiang
    IEEE ACCESS, 2020, 8 : 139670 - 139679
  • [5] Joint Trajectory Control, Frequency Allocation, and Routing for UAV Swarm Networks: A Multi-Agent Deep Reinforcement Learning Approach
    Alam, Muhammad Morshed
    Moh, Sangman
    IEEE TRANSACTIONS ON MOBILE COMPUTING, 2024, 23 (12) : 11989 - 12005
  • [6] Decentralized Trajectory and Power Control Based on Multi-Agent Deep Reinforcement Learning in UAV Networks
    Chen, Binqiang
    Liu, Dong
    Hanzo, Lajos
    IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC 2022), 2022, : 3983 - 3988
  • [7] Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks
    Cui, Jingjing
    Liu, Yuanwei
    Nallanathan, Arumugam
    IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2020, 19 (02) : 729 - 743
  • [8] Power Allocation and Energy Cooperation for UAV-Enabled MmWave Networks: A Multi-Agent Deep Reinforcement Learning Approach
    Domingo, Mari Carmen
    SENSORS, 2022, 22 (01)
  • [9] Joint Resource Allocation on Slot, Space and Power Towards Concurrent Transmissions in UAV Ad Hoc Networks
    Wang, Haijun
    Jiang, Bo
    Zhao, Haitao
    Zhang, Jiao
    Zhou, Li
    Ma, Dongtang
    Wei, Jibo
    Leung, Victor C. M.
    IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2022, 21 (10) : 8698 - 8712
  • [10] Resource Allocation in UAV-D2D Networks: A Scalable Heterogeneous Multi-Agent Deep Reinforcement Learning Approach
    Wang, Huayuan
    Li, Hui
    Wang, Xin
    Xia, Shilin
    Liu, Tao
    Wang, Ruonan
    ELECTRONICS, 2024, 13 (22)