Value functions and performance evaluation in stochastic network models

This paper concerns control and performance evaluation for stochastic network models. Structural properties of value functions are developed for controlled random-walk (CRW) models; and associated controlled Brownian motion (CBM) and deterministic (fluid) workload-models. Based on these results we obtain the following conclusions: Outside of a null-set of network parameters, (i) The fluid value function is continuously differentiable. Under further minor conditions, the fluid value function satisfies the Neumann boundary conditions that are required to ensure it solves a martingale problem for the CBM model. (ii) The fluid value function provides a shadow function for use in simulation variance reduction for CRW models. The resulting simulator satisfies an exact large deviation principle, while a standard simulation algorithm does not satisfy any such bound. (iii) The fluid value function provides upper and lower bounds on performance for the CRW and CBM models. This follows from an extension of recent linear programming approaches to performance evaluation.