Joint Resource Scheduling of the Time Slot, Power, and Main Lobe Direction in Directional UAV Ad Hoc Networks: A Multi-Agent Deep Reinforcement Learning Approach

被引:0
|
作者
Liang, Shijie [1 ,2 ]
Zhao, Haitao [2 ]
Zhou, Li [2 ]
Wang, Zhe [2 ]
Cao, Kuo [2 ]
Wang, Junfang [1 ]
机构
[1] China Elect Technol Grp Corp, Res Inst 54, Shijiazhuang 050081, Peoples R China
[2] Natl Univ Def Technol, Coll Elect Sci & Technol, Changsha 410073, Peoples R China
基金
中国国家自然科学基金;
关键词
directional UAV ad hoc network; resource scheduling; multi-agent deep reinforcement learning; attention mechanism; transmission fairness; ALLOCATION; COMMUNICATION; ACCESS;
D O I
10.3390/drones8090478
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
Directional unmanned aerial vehicle (UAV) ad hoc networks (DUANETs) are widely applied due to their high flexibility, strong anti-interference capability, and high transmission rates. However, within directional networks, complex mutual interference persists, necessitating scheduling of the time slot, power, and main lobe direction for all links to improve the transmission performance of DUANETs. To ensure transmission fairness and the total count of transmitted data packets for the DUANET under dynamic data transmission demands, a scheduling algorithm for the time slot, power, and main lobe direction based on multi-agent deep reinforcement learning (MADRL) is proposed. Specifically, modeling is performed with the links as the core, optimizing the time slot, power, and main lobe direction variables for the fairness-weighted count of transmitted data packets. A decentralized partially observable Markov decision process (Dec-POMDP) is constructed for the problem. To process the observation in Dec-POMDP, an attention mechanism-based observation processing method is proposed to extract observation features of UAVs and their neighbors within the main lobe range, enhancing algorithm performance. The proposed Dec-POMDP and MADRL algorithms enable distributed autonomous decision-making for the resource scheduling of time slots, power, and main lobe directions. Finally, the simulation and analysis are primarily focused on the performance of the proposed algorithm and existing algorithms across varying data packet generation rates, different main lobe gains, and varying main lobe widths. The simulation results show that the proposed attention mechanism-based MADRL algorithm enhances the performance of the MADRL algorithm by 22.17%. The algorithm with the main lobe direction scheduling improves performance by 67.06% compared to the algorithm without the main lobe direction scheduling.
引用
收藏
页数:27
相关论文
共 50 条
  • [31] Real-time production scheduling using a deep reinforcement learning-based multi-agent approach
    Taghipour, Sharareh
    Namoura, Hamed A.
    Sharifi, Mani
    Ghaleb, Mageed
    INFOR, 2024, 62 (02) : 186 - 210
  • [32] Weighted fair energy transfer in a UAV network: A multi-agent deep reinforcement learning approach
    Murshed, Shabab
    Nibir, Abu Shaikh
    Razzaque, Md. Abdur
    Roy, Palash
    Elhendi, Ahmed Zohier
    Hassan, Md. Rafiul
    Hassan, Mohammad Mehedi
    ENERGY, 2024, 292
  • [33] Multi-Agent Low-Bias Reinforcement Learning for Resource Allocation in UAV-Assisted Networks
    Zhou, Shiyang
    Cheng, Yufan
    Lei, Xia
    2022 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS WORKSHOPS (ICC WORKSHOPS), 2022, : 1011 - 1016
  • [34] Joint Secure Offloading and Resource Allocation for Vehicular Edge Computing Network: A Multi-Agent Deep Reinforcement Learning Approach
    Ju, Ying
    Chen, Yuchao
    Cao, Zhiwei
    Liu, Lei
    Pei, Qingqi
    Xiao, Ming
    Ota, Kaoru
    Dong, Mianxiong
    Leung, Victor C. M.
    IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2023, 24 (05) : 5555 - 5569
  • [35] Multi-Agent Deep Reinforcement Learning-Based Task Scheduling and Resource Sharing for O-RAN-Empowered Multi-UAV-Assisted Wireless Sensor Networks
    Betalo, Mesfin Leranso
    Leng, Supeng
    Abishu, Hayla Nahom
    Dharejo, Fayaz Ali
    Seid, Abegaz Mohammed
    Erbad, Aiman
    Naqvi, Rizwan Ali
    Zhou, Longyu
    Guizani, Mohsen
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2024, 73 (07) : 9247 - 9261
  • [36] Multi-Agent Reinforcement Learning for Joint Cooperative Spectrum Sensing and Channel Access in Cognitive UAV Networks
    Jiang, Weiheng
    Yu, Wanxin
    Wang, Wenbo
    Huang, Tiancong
    SENSORS, 2022, 22 (04)
  • [37] Joint UAV Trajectory and RadCom Task Schedule for IVNs: A Game-Embedding Multi-Agent Deep Reinforcement Learning Approach
    Cheng, Sike
    Lin, Xiangbo
    Li, Xuanheng
    Wang, Jingjing
    IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2025, 24 (01) : 181 - 196
  • [38] Orchestrated Scheduling and Multi-Agent Deep Reinforcement Learning for Cloud-Assisted Multi-UAV Charging Systems
    Jung, Soyi
    Yun, Won Joon
    Shin, MyungJae
    Kim, Joongheon
    Kim, Jae-Hyun
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2021, 70 (06) : 5362 - 5377
  • [39] Multi-Agent Deep Reinforcement Learning for Distributed Resource Management in Wirelessly Powered Communication Networks
    Hwang, Sangwon
    Kim, Hanjin
    Lee, Hoon
    Lee, Inkyu
    IEEE Transactions on Vehicular Technology, 2020, 69 (11): : 14055 - 14060
  • [40] Resource allocation strategy for vehicular communication networks based on multi-agent deep reinforcement learning
    Liu, Zhibin
    Deng, Yifei
    VEHICULAR COMMUNICATIONS, 2025, 53