VALUE ITERATION IN COUNTABLE STATE AVERAGE COST MARKOV DECISION PROCESSES WITH UNBOUNDED COSTS

We deal with countable state Markov decision processes with finite action sets and (possibly) unbounded costs. Assuming the existence of an expected average cost optimal stationary policy f, with expected average cost g, when can f and g be found using undiscounted value iteration? We give assumptions guaranteeing the convergence of a quantity related to ng-v(n)(i), where v(n)(i) is the minimum expected n-stage cost when the process starts in state i. The theory is applied to a queueing system with variable service rates and to a queueing system with variable arrival parameter.