Improving strategies in stochastic games

In a zero-sum limiting average stochastic game, we evaluate a strategy pi for the maximizing player, player 1, by the reward phi(s)(pi) that pi guarantees to him when starting in state s. A strategy pi is called non-improving if phi(s)(pi) greater than or equal to phi(s)(pi[h]) for any state s and for any finite history h, where pi[h] is the strategy pi conditional on the history h; otherwise the strategy is called improving. We investigate the use of improving and non-improving strategies, and explore the relation between (non-)improvingness and (epsilon-)optimality. Improving strategies appear to play a very important role for obtaining E-optimality, while 0-optimal strategies are always non-improving. Several examples will clarify all these issues.