A CLASS OF SELF-INTERACTING PROCESSES WITH APPLICATIONS TO GAMES AND REINFORCED RANDOM WALKS

This paper studies a class of non-Markovian and nonhomogeneous stochastic processes on a finite state space. Relying on a recent paper by Benaim, Hofbauer, and Sorin [SIAM J. Control Optim., 44 (2005), pp. 328-348] it is shown that, under certain assumptions, the asymptotic behavior of occupation measures can be described in terms of a certain set-valued deterministic dynamical system. This provides a unified approach to simulated annealing type processes and permits the study of new models of vertex reinforced random walks and new models of learning in games such as Markovian fictitious play.