Iterative algorithms for solving one-sided partially observable stochastic shortest path games

被引：0
|
作者：
Tomášek, Petr [1 ]
Horák, Karel [1 ]
Bošanský, Branislav [1 ]
机构：
[1] Artificial Intelligence Center, Dept. of Computer Science, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic
来源：
International Journal of Approximate Reasoning | 2024年 / 175卷
关键词：
Real-world scenarios often involve dynamic interactions among competing agents; where decisions are made considering actions taken by others. These situations can be modeled as partially observable stochastic games (POSGs); with zero-sum variants capturing strictly competitive interactions (e.g; security scenarios). While such models address a broad range of problems; they commonly focus on infinite-horizon scenarios with discounted-sum objectives. Using the discounted-sum objective; however; can lead to suboptimal solutions in cases where the length of the interaction does not directly affect the gained rewards of the players. We thus focus on games with undiscounted objective and an indefinite horizon where every realization of the game is guaranteed to terminate after some unspecified number of turns. To manage the computational complexity of solving POSGs in general; we restrict to games with one-sided partial observability where only one player has imperfect information while their opponent is provided with full information about the current situation. We introduce two novel algorithms based on the heuristic search value iteration (HSVI) algorithm that iteratively solve sequences of easier-to-solve approximations of the game using fundamentally different approaches for constructing the sequences: (1) in GoalHorizon; the game approximations are based on a limited number of turns in which players can change their actions; (2) in GoalDiscount; the game approximations are constructed using an increasing discount factor. We provide theoretical qualitative guarantees for algorithms; and we also experimentally demonstrate that these algorithms are able to find near-optimal solutions on pursuit-evasion games and a game modeling privilege escalation problem from computer security. © 2024 Elsevier Inc;
D O I：
10.1016/j.ijar.2024.109297
中图分类号：
学科分类号：
摘要：
引用
收藏
相关论文
共 50 条
  • [1] Solving zero-sum one-sided partially observable stochastic games
    Horak, Karel
    Bosansky, Branislav
    Kovarik, Vojtech
    Kiekintveld, Christopher
    ARTIFICIAL INTELLIGENCE, 2023, 316
  • [2] Solving Partially Observable Stochastic Shortest-Path Games
    Tomasek, Petr
    Horak, Karel
    Aradhye, Aditya
    Bosansky, Branislav
    Chatterjee, Krishnendu
    PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 4182 - 4189
  • [3] Heuristic Search Value Iteration for One-Sided Partially Observable Stochastic Games
    Horak, Karel
    Bosansky, Branislav
    Pechoucek, Michal
    THIRTY-FIRST AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2017, : 558 - 564
  • [4] The Stackelberg equilibrium for one-sided zero-sum partially observable stochastic games
    Zheng, Wei
    Jung, Taeho
    Lin, Hai
    AUTOMATICA, 2022, 140
  • [5] Dynamic Programming for One-sided Partially Observable Pursuit-evasion Games
    Horak, Karel
    Bosansky, Branislav
    ICAART: PROCEEDINGS OF THE 9TH INTERNATIONAL CONFERENCE ON AGENTS AND ARTIFICIAL INTELLIGENCE, VOL 2, 2017, : 503 - 510
  • [6] Solving Partially Observable Stochastic Games with Public Observations
    Horak, Karel
    Bosansky, Branislav
    THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, : 2029 - 2036
  • [7] ON ONE-SIDED STOCHASTIC GAMES AND THEIR APPLICATIONS TO FINANCE
    Dshalalow, Jewgeni H.
    Robinson, Randy
    STOCHASTIC MODELS, 2012, 28 (01) : 1 - 14
  • [8] A Point-Based Approximate Algorithm for One-Sided Partially Observable Pursuit-Evasion Games
    Horak, Karel
    Bosansky, Branislav
    DECISION AND GAME THEORY FOR SECURITY, (GAMESEC 2016), 2016, 9996 : 435 - 454
  • [9] Stochastic shortest path games
    Patek, SD
    Bertsekas, DP
    SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 1999, 37 (03) : 804 - 824
  • [10] An Iterative Method for Solving Relaxed One-Sided Lipschitz Algebraic Inclusions
    Beyn, Wolf-Juergen
    Rieger, Janosch
    JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 2015, 164 (01) : 154 - 172
12345