Joint Resource Scheduling of the Time Slot, Power, and Main Lobe Direction in Directional UAV Ad Hoc Networks: A Multi-Agent Deep Reinforcement Learning Approach

被引:0
|
作者
Liang, Shijie [1 ,2 ]
Zhao, Haitao [2 ]
Zhou, Li [2 ]
Wang, Zhe [2 ]
Cao, Kuo [2 ]
Wang, Junfang [1 ]
机构
[1] China Elect Technol Grp Corp, Res Inst 54, Shijiazhuang 050081, Peoples R China
[2] Natl Univ Def Technol, Coll Elect Sci & Technol, Changsha 410073, Peoples R China
基金
中国国家自然科学基金;
关键词
directional UAV ad hoc network; resource scheduling; multi-agent deep reinforcement learning; attention mechanism; transmission fairness; ALLOCATION; COMMUNICATION; ACCESS;
D O I
10.3390/drones8090478
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
Directional unmanned aerial vehicle (UAV) ad hoc networks (DUANETs) are widely applied due to their high flexibility, strong anti-interference capability, and high transmission rates. However, within directional networks, complex mutual interference persists, necessitating scheduling of the time slot, power, and main lobe direction for all links to improve the transmission performance of DUANETs. To ensure transmission fairness and the total count of transmitted data packets for the DUANET under dynamic data transmission demands, a scheduling algorithm for the time slot, power, and main lobe direction based on multi-agent deep reinforcement learning (MADRL) is proposed. Specifically, modeling is performed with the links as the core, optimizing the time slot, power, and main lobe direction variables for the fairness-weighted count of transmitted data packets. A decentralized partially observable Markov decision process (Dec-POMDP) is constructed for the problem. To process the observation in Dec-POMDP, an attention mechanism-based observation processing method is proposed to extract observation features of UAVs and their neighbors within the main lobe range, enhancing algorithm performance. The proposed Dec-POMDP and MADRL algorithms enable distributed autonomous decision-making for the resource scheduling of time slots, power, and main lobe directions. Finally, the simulation and analysis are primarily focused on the performance of the proposed algorithm and existing algorithms across varying data packet generation rates, different main lobe gains, and varying main lobe widths. The simulation results show that the proposed attention mechanism-based MADRL algorithm enhances the performance of the MADRL algorithm by 22.17%. The algorithm with the main lobe direction scheduling improves performance by 67.06% compared to the algorithm without the main lobe direction scheduling.
引用
收藏
页数:27
相关论文
共 50 条
  • [21] A multi-objective multi-agent deep reinforcement learning approach to residential appliance scheduling
    Lu, Junlin
    Mannion, Patrick
    Mason, Karl
    IET SMART GRID, 2022, 5 (04) : 260 - 280
  • [22] Multi-NPDQ: A Multi-agent Approach Through Deep Reinforcement Learning for Operation Scheduling
    Yao, Linwei
    Chen, Qichao
    Gong, Lianghao
    Li, Kuan
    ADVANCED INTELLIGENT COMPUTING TECHNOLOGY AND APPLICATIONS, PT II, ICIC 2024, 2024, 14863 : 467 - 479
  • [23] Multi-agent deep reinforcement learning approach for EV charging scheduling in a smart grid
    Park, Keonwoo
    Moon, Ilkyeong
    APPLIED ENERGY, 2022, 328
  • [24] Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach
    Sharma, Mohit K.
    Zappone, Alessio
    Assaad, Mohamad
    Debbah, Merouane
    Vassilaras, Spyridon
    IEEE TRANSACTIONS ON COGNITIVE COMMUNICATIONS AND NETWORKING, 2019, 5 (04) : 1140 - 1154
  • [25] Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
    Emami, Yousef
    Wei, Bo
    Li, Kai
    Ni, Wei
    Tovar, Eduardo
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2021, 70 (10) : 10986 - 10998
  • [26] Safe multi-agent deep reinforcement learning for joint bidding and maintenance scheduling of generation units
    Rokhforoz, Pegah
    Montazeri, Mina
    Fink, Olga
    RELIABILITY ENGINEERING & SYSTEM SAFETY, 2023, 232
  • [27] Using Statistical Network Link Model for Routing in Ad Hoc Networks with multi-agent Reinforcement Learning
    Zhang Binbin
    Liu Quan
    Zhao Shouling
    2ND IEEE INTERNATIONAL CONFERENCE ON ADVANCED COMPUTER CONTROL (ICACC 2010), VOL. 3, 2010, : 462 - 466
  • [28] A multi-agent deep reinforcement learning approach for optimal resource management in serverless computing
    Singh, Ashutosh Kumar
    Kumar, Satender
    Jain, Sarika
    CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2025, 28 (02):
  • [29] Multi-Agent Power and Resource Allocation for D2D Communications: A Deep Reinforcement Learning Approach
    Xiang, Honglin
    Peng, Jingyi
    Gao, Zhen
    Li, Lingjie
    Yang, Yang
    2022 IEEE 96TH VEHICULAR TECHNOLOGY CONFERENCE (VTC2022-FALL), 2022,
  • [30] Joint User Scheduling and Beam Selection in mmWave Networks Based on Multi-Agent Reinforcement Learning
    Xu, Chunmei
    Liu, Shengheng
    Zhang, Cheng
    Huang, Yongming
    Yang, Luxi
    2020 IEEE 11TH SENSOR ARRAY AND MULTICHANNEL SIGNAL PROCESSING WORKSHOP (SAM), 2020,