Combining the stochastic counterpart and stochastic approximation methods

In this work, we examine how to combine the score function method with the standard crude Monte Carlo and experimental design approaches, in order to evaluate the expected performance of a discrete event system and its associated gradient simultaneously for different scenarios (combinations of parameter values), as well as to optimize the expected performance with respect to two parameter sets, which represent parameters of the underlying probability law (for the system's evolution) and parameters of the sample performance measure, respectively. We explore how the stochastic approximation and stochastic counterpart methods can be combined to perform optimization with respect to both sets of parameters at the same time. We outline three combined algorithms of that form, one sequential and two parallel, and give a convergence proof for one of them. We discuss a number of issues related to the implementation and convergence of those algorithms, introduce averaging variants, and give numerical illustrations.