Optimal stopping and Gittins' indices for piecewise deterministic evolution processes

We study the optimal stopping problem for a class of continuous time random evolutions described by stochastic differential equations with alternating renewal processes as noise sources. The exact solution of this stopping problem provides, in explicit form, an expression for the Gittins' indices needed to derive the optimal scheduling of a class of multi-armed bandit problems in continuous time. The underlying random processes to which the bandits' arms obey are random velocity models. Such processes are commonly used to describe, in the fluid limit, the random production flows delivered by failure prone machines.