A robust gradient sampling algorithm for nonsmooth, nonconvex optimization

Let f be a continuous function on R-n, and suppose f is continuously differentiable on an open dense subset. Such functions arise in many applications, and very often minimizers are points at which f is not differentiable. Of particular interest is the case where f is not convex, and perhaps not even locally Lipschitz, but is a function whose gradient is easily computed where it is defined. We present a practical, robust algorithm to locally minimize such functions, based on gradient sampling. No subgradient information is required by the algorithm. When f is locally Lipschitz and has bounded level sets, and the sampling radius epsilon is fixed, we show that, with probability 1, the algorithm generates a sequence with a cluster point that is Clarke epsilon-stationary. Furthermore, we show that if f has a unique Clarke stationary point (x) over bar, then the set of all cluster points generated by the algorithm converges to (x) over bar as epsilon is reduced to zero. Numerical results are presented demonstrating the robustness of the algorithm and its applicability in a wide variety of contexts, including cases where f is not locally Lipschitz at minimizers. We report approximate local minimizers for functions in the applications literature which have not, to our knowledge, been obtained previously. When the termination criteria of the algorithm are satisfied, a precise statement about nearness to Clarke epsilon-stationarity is available. A matlab implementation of the algorithm is posted at http://www.cs.nyu.edu/ overton/papers/gradsamp/alg.