Pure strategy Markov equilibrium in stochastic games with a continuum of players

We discuss stochastic games which are played by a continuum of players. The player space is given by the atomless measure space of a unit interval with the Lebesgue measure. The state space in period t is given by (S, Sigma, mu(t)), where S is a complete, separable metric space, Sigma is the Borel sigma algebra and mu(t) is a fixed probability measure. There is a common action space K which is either a finite set or the K-ball of an Euclidean space R-n. The transition probabilities in period t are product measurable in the realizations of the state and average responses and norm continuous in the current average response of the players and absolutely continuous with respect to the measure mu(t). The payoffs of the players in these infinite horizon stochastic games are the discounted sum of the payoffs received every period. We show that these stochastic games have pure strategy Markov equilibrium points. In the process we are able to provide a characterization of Markov equilibrium strategy as the equilibrium strategies of a sequence of single-period constituent games. (C) 2003 Elsevier Science B.V. All rights reserved.