A framework for simulation-optimization software

Research on the optimization of stochastic systems via simulation often centers on the development of algorithms for which global convergence can be guaranteed. On the other hand, commercial software applications that perform optimization via simulation typically employ search heuristics that have been successful in deterministic settings. Such search heuristics give up on global convergence in order to be more generally applicable and to yield rapid progress towards good solutions. Unfortunately, commercial applications do not always formally account for the randomness in simulation responses, meaning that their progress may be no better than a random search if the variability of the outputs is high. In addition, they do not provide statistical guarantees about the "goodness" of the final results. In practice, simulation studies often rely heavily on engineers who, in addition to developing the simulation model and generating the alternatives to be compared, must also perform the statistical analyses off-line. This is a time- and labor-consuming process. In this paper, we report on the work we have done to implement statistical error control within a heuristic search procedure, and on our automated procedure to deliver a statistical guarantee after the search procedure is finished. We describe how we implemented these techniques in software developed for JGC Corporation of Japan.