Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning

被引：0

作者：

Liu R. ^{[1
]}

Ma T. ^{[1
]}

Wu W. ^{[2
]}

Yao C. ^{[1
]}

Yang Q. ^{[3
]}

机构：

[1] School of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an

[2] School of Physics and Information Technology, Shaanxi Normal University, Xi’an

[3] School of Telecommunications Engineering, Xidian University, Xi’an

来源：

Tongxin Xuebao/Journal on Communications | 2023年 / 44卷 / 07期

基金：

中国国家自然科学基金;

关键词：

deep reinforcement learning; dynamic scheduling; multi-objective optimization; relay satellite networks; task scheduling;

D O I：

10.11959/j.issn.1000-436x.2023130

中图分类号：

学科分类号：

摘要：

In recent years, with the increasing number of various emergency tasks, how to control the impact on common tasks while ensuring system revenue has become a huge challenge for the dynamic scheduling of relay satellite networks. Aiming at this problem, with the goal of maximizing the total revenue of emergency tasks and minimizing the damage to common tasks, a dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning was proposed. Specifically, in order to take into account the long-term and short-term performance of the system at the same time, a two-layer scheduling framework implemented by upper-level and lower-level DQN was designed. The upper-level DQN was responsible for determining the temporary optimization goal based on long-term performance, and the lower-level DQN determined the scheduling strategy for current task according to the optimization goal. Simulation results show that compared with traditional deep learning methods and the heuristic methods dealing with dynamic scheduling problems, the proposed method can improve the total revenue of urgent tasks while reducing the damage to common tasks. © 2023 Editorial Board of Journal on Communications. All rights reserved.

引用

页码：207 / 217

页数：10

共 15 条

[1] WANG J., China’s data relay satellite system and its application prospect, Spacecraft Engineering, 22, 1, pp. 1-6, (2013)
[2] HE C, ZHU X M, QIU D S., Cooperative scheduling method of multi-satellites for imaging reconnaissance in emergency condition, Systems Engineering and Electronics, 34, 4, pp. 726-731, (2012)
[3] WU G H, MA M H, ZHU J H, Et al., Multi-satellite observation integrated scheduling method oriented to emergency tasks and common tasks, Journal of Systems Engineering and Electronics, 23, 5, pp. 723-733, (2012)
[4] LI F L, LI G X, LI Z Q, Et al., Topology structure and routing policy based on multilayered clusters in Beidou satellite navigation system, Journal on Communications, 35, 10, pp. 31-41, (2014)
[5] HAN S M, BEAK S W, CHO K R, Et al., Satellite mission scheduling using genetic algorithm, Proceedings of 2008 SICE Annual Conference, pp. 1226-1230, (2008)
[6] HUI-CHCNG H, JIANG W, LI Y J., Solving on agile satellites mission planning based on tabu search-parallel genetic algorithms, Proceed ings of 2013 International Conference on Management Science and Proceedings of Engineering 20th Annual Conference Proceedings, pp. 120-125, (2013)
[7] HE L J, LI J D, SHENG M, Et al., Dynamic scheduling of hybrid tasks with time windows in data relay satellite networks, IEEE Transactions on Vehicular Technology, 68, 5, pp. 4989-5004, (2019)
[8] DENG B Y, JIANG C X, KUANG L L, Et al., Two-phase task scheduling in data relay satellite systems, IEEE Transactions on Vehicular Technology, 67, 2, pp. 1782-1793, (2018)
[9] DAI C Q, LI C, FU S, Et al., Dynamic scheduling for emergency tasks in space data relay network, IEEE Transactions on Vehicular Technology, 70, 1, pp. 795-807, (2021)
[10] ZHANG Z J, ZHANG N, FENG Z R., Multi-satellite control resource scheduling based on ant colony optimization, Expert Systems with Applications, 41, 6, pp. 2816-2823, (2014)

← 1 2 →