On the speed of convergence of value iteration on stochastic shortest-path problems

We establish a bound on the convergence time of the value iteration algorithm on stochastic shortest-path problems. The bound. which applies for admissible initial vectors as, for example, J 0, implies a polynomial-time convergence of value iteration for all problems with polynormally bounded ||J*||/g. This result gives a partial answer to the open problem of bounding the convergence time of value iteration on arbitrary initial vectors. The proof is obtained by analyzing a stochastic process associated with the shortest-path problem.