Q-Learning in Regularized Mean-field Games

被引:17
|
作者
Anahtarci, Berkay [1 ]
Kariksiz, Can Deha [1 ]
Saldi, Naci [2 ]
机构
[1] Ozyegin Univ, Istanbul, Turkey
[2] Bilkent Univ, Ankara, Turkey
关键词
Mean-field games; Q-learning; Regularized Markov decision processes; Discounted reward; NASH EQUILIBRIA; DYNAMIC-GAMES; ROBUSTNESS;
D O I
10.1007/s13235-022-00450-2
中图分类号
O1 [数学];
学科分类号
0701 ; 070101 ;
摘要
In this paper, we introduce a regularized mean-field game and study learning of this game under an infinite-horizon discounted reward function. Regularization is introduced by adding a strongly concave regularization function to the one-stage reward function in the classical mean-field game model. We establish a value iteration based learning algorithm to this regularized mean-field game using fitted Q-learning. The regularization term in general makes reinforcement learning algorithm more robust to the system components. Moreover, it enables us to establish error analysis of the learning algorithm without imposing restrictive convexity assumptions on the system components, which are needed in the absence of a regularization term.
引用
收藏
页码:89 / 117
页数:29
相关论文
共 50 条
  • [41] Stationary mean-field games with logistic effects
    Gomes, Diogo Aguiar
    Ribeiro, Ricardo de Lima
    PARTIAL DIFFERENTIAL EQUATIONS AND APPLICATIONS, 2021, 2 (01):
  • [42] Asynchronous Decentralized Q-Learning in Stochastic Games
    Yongacoglu, Bora
    Arslan, Gurdal
    Yuksel, Serdar
    2023 62ND IEEE CONFERENCE ON DECISION AND CONTROL, CDC, 2023, : 5008 - 5013
  • [43] On the Efficiency of Equilibria in Mean-field Oscillator Games
    Yin, Huibing
    Mehta, Prashant G.
    Meyn, Sean P.
    Shanbhag, Uday V.
    2011 AMERICAN CONTROL CONFERENCE, 2011, : 5354 - 5359
  • [44] Decentralized Q-Learning for Stochastic Teams and Games
    Arslan, Gurdal
    Yuksel, Serdar
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2017, 62 (04) : 1545 - 1558
  • [45] Mean-field games and green power control
    Laboratoire des Signaux et Systèmes , Supélec, 3 rue Joliot-Curie, 91192 Gif-sur-Yvette Cedex, France
    Int. Conf. NETw. Games, Control Optim., NetGCooP,
  • [46] Nonlinear elliptic systems and mean-field games
    Bardi, Martino
    Feleqi, Ermal
    NODEA-NONLINEAR DIFFERENTIAL EQUATIONS AND APPLICATIONS, 2016, 23 (04):
  • [47] On the Efficiency of Equilibria in Mean-Field Oscillator Games
    Huibing Yin
    Prashant G. Mehta
    Sean P. Meyn
    Uday V. Shanbhag
    Dynamic Games and Applications, 2014, 4 : 177 - 207
  • [48] Risk-Sensitive Mean-Field Games
    Tembine, Hamidou
    Zhu, Quanyan
    Basar, Tamer
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2014, 59 (04) : 835 - 850
  • [49] LQG Mean-Field Games with ergodic cost
    Bardi, Martino
    Priuli, Fabio S.
    2013 IEEE 52ND ANNUAL CONFERENCE ON DECISION AND CONTROL (CDC), 2013, : 2493 - 2498
  • [50] Stationary fully nonlinear mean-field games
    Andrade, Pedra D. S.
    Pimentel, Edgard A.
    JOURNAL D ANALYSE MATHEMATIQUE, 2021, 145 (01): : 335 - 356