Continuous-Time Stochastic Games of Fixed Duration

We study nonzero-sum continuous-time stochastic games, also known as continuous-time Markov games, of fixed duration. We concentrate on Markovian strategies. We show by way of example that equilibria need not exist in Markovian strategies, but they always exist in Markovian public-signal correlated strategies. To do so, we develop criteria for a strategy profile to be an equilibrium via differential inclusions, both directly and also by modeling continuous-time stochastic as differential games and using the Hamilton–Jacobi–Bellman equations. We also give an interpretation of equilibria in mixed strategies in continuous time and show that approximate equilibria always exist.