Adaptive policies for time-varying stochastic systems under discounted criterion

We consider a class of time-varying stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to a discrete-time equation x(n+1) = G(n)(x(n), a(n), xi(n)), n = 0, 1,..., where the xi(n) are i.i.d. R-k-valued random vectors whose common density is unknown, and the G, are given functions converging, in a restricted way, to some function Ginfinity as n --> infinity. Assuming observability of xi(n), we construct an adaptive policy which is asymptotically discounted cost optimal for the limiting control system x(n+1) = Ginfinity(x(n), a(n), xi(n)).