The Fast and the Robust: Trade-Offs Between Optimization Robustness and Cost in the Calibration of Environmental Models

Environmental modelers using optimization algorithms for model calibration face an ambivalent choice. Some algorithms, for example, Newton-type methods, are fast but struggle to consistently find global parameter optima; other algorithms, for example, evolutionary methods, boast better global convergence but at much higher cost (e.g., requiring more objective function calls). Trade-offs between accuracy/robustness versus cost are ubiquitous in numerical computation, yet environmental modeling studies have lacked a systematic framework for quantifying these trade-offs. This study develops a framework for benchmarking stochastic optimization algorithms in the context of environmental model calibration, where multiple algorithm invocations are typically necessary. We define reliability as the probability of finding the desired (global or tolerable) optimum from random initial points and estimate the number of invocations to find the desired optimum with prescribed confidence (here 95%). A robust algorithm should achieve consistently high reliability across many problems. A characteristic efficiency metric for algorithm benchmarking is defined as the total cost (objective function calls over multiple invocations) to find the desired optimum with prescribed confidence. This approach avoids the pitfalls of existing approaches that compare costs without controlling the confidence in algorithm success. A case study illustrates the framework by benchmarking the Levenberg-Marquardt and Shuffled Complex Evolution (SCE) algorithms over three catchments and four hydrological models. In 8 of 12 scenarios, Levenberg-Marquardt is more efficient than SCEby sacrificing some of its speed advantage to match SCE reliability through more invocations. The proposed framework is easy to apply and can help guide algorithm selection in environmental model calibration.