Monomial Strategies for Concurrent Reachability Games and Other Stochastic Games

We consider two-player zero-sum finite (but infinite-horizon) stochastic games with limiting average payoffs. We define a family of stationary strategies for Player I parameterized by e > 0 to be monomial, if for each state k and each action j of Player I in state k except possibly one action, we have that the probability of playing j in k is given by an expression of the form ced for some non-negative real number c and some non-negative integer d. We show that for all games, there is a monomial family of stationary strategies that are e-optimal among stationary strategies. A corollary is that all concurrent reachability games have a monomial family of e-optimal strategies. This generalizes a classical result of de Alfaro, Henzinger and Kupferman who showed that this is the case for concurrent reachability games where all states have value 0 or 1.