Online Policy Iteration Algorithm for Semi-Markov Switching State-Space Control Processes

被引：0
|
作者：
Jiang, Qi [1 ]
Xi, Hong-Sheng [2 ]
Yin, Bao-Qin [2 ]
机构：
[1] Hefei Univ Technol, Dept Automat, Hefei 230009, Peoples R China
[2] Univ Sci & Technol China, Dept Automat, Hefei 230027, Peoples R China
来源：
PROCEEDINGS OF THE 48TH IEEE CONFERENCE ON DECISION AND CONTROL, 2009 HELD JOINTLY WITH THE 2009 28TH CHINESE CONTROL CONFERENCE (CDC/CCC 2009) | 2009年
基金：
中国国家自然科学基金; 中国博士后科学基金;
关键词：
DECISION-PROCESSES; SENSITIVITY-ANALYSIS; OPTIMIZATION; CONVERGENCE; POTENTIALS; SYSTEMS;
D O I：
10.1109/CDC.2009.5400958
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
An event-based online policy iteration algorithm is presented for addressing hierarchical optimization problems. First, an event-driven analytical model with dynamic hierarchy called semi-Markov switching state-space control processes is introduced. Then, by exploiting the structure of dynamic hierarchy and the features of event-driven policy, an online adaptive optimization algorithm that combines potentials estimation and policy iteration is proposed. The convergence of this algorithm is also proved. Finally, as an illustrative example, the dynamic service composition in a service overlay network is formulated and addressed. Simulation results demonstrate the effectiveness of the presented algorithm.
引用
收藏
页码：2298 / 2303
页数：6
相关论文
共 50 条
  • [31] Optimum maintenance policy with inspection by Semi-Markov Decision Processes
    Ge, Haifeng
    Tomasevicz, Curtis L.
    Asgarpoor, Sohrab
    [J]. 2007 39TH NORTH AMERICAN POWER SYMPOSIUM, VOLS 1 AND 2, 2007, : 541 - 546
  • [32] Optimum maintenance policy using semi-Markov decision processes
    Tomasevicz, Curtis L.
    Asgarpoor, Sohrab
    [J]. 2006 38TH ANNUAL NORTH AMERICAN POWER SYMPOSIUM, NAPS-2006 PROCEEDINGS, 2006, : 23 - +
  • [33] Deterministic policy gradient algorithms for semi-Markov decision processes
    Hosseinloo, Ashkan Haji
    Dahleh, Munther A.
    [J]. INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, 2022, 37 (07) : 4008 - 4019
  • [34] An approximation approach to ergodic semi-Markov control processes
    Jaskiewicz, A
    [J]. MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 2001, 54 (01) : 1 - 19
  • [35] Optimum maintenance policy using semi-Markov decision processes
    Tomasevicz, Curtis L.
    Asgarpoor, Sohrab
    [J]. ELECTRIC POWER SYSTEMS RESEARCH, 2009, 79 (09) : 1286 - 1291
  • [36] ON THE THEORY OF SEMI-MARKOV PROCESSES WITH COMMON PHASE-SPACE
    TOMKO, J
    [J]. THEORY OF PROBABILITY AND ITS APPLICATIONS, 1989, 34 (02) : 275 - 289
  • [37] An Inverse Reinforcement Learning Algorithm for semi-Markov Decision Processes
    Tan, Chuanfang
    Li, Yanjie
    Cheng, Yuhu
    [J]. 2017 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (SSCI), 2017, : 1256 - 1261
  • [38] Discrete filtration of stochastic processes with random structure with semi-Markov switching
    Zhuk, SY
    [J]. IZVESTIYA VYSSHIKH UCHEBNYKH ZAVEDENII RADIOELEKTRONIKA, 1995, 38 (7-8): : A21 - A30
  • [39] Policy Iteration for Decentralized Control of Markov Decision Processes
    Bernstein, Daniel S.
    Amato, Christopher
    Hansen, Eric A.
    Zilberstein, Shlomo
    [J]. JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, 2009, 34 : 89 - 132
  • [40] REMARK CONCERNING 2-STATE SEMI-MARKOV PROCESSES
    DERMAN, C
    [J]. ANNALS OF MATHEMATICAL STATISTICS, 1961, 32 (02): : 615 - 616
12345