Impact of non-smooth observation operators on variational and sequential data assimilation for a limited-area shallow-water equation model

We investigate the issue of variational and sequential data assimilation with nonlinear and non-smooth observation operators using a two-dimensional limited-area shallow-water equation model and its adjoint. The performance of the four-dimensional variational approach (4D-Var: two dimensions plus time) compared with that of the maximum-likelihood ensemble filter (MLEF), a hybrid ensemble/variational method, is tested in the presence of non-smooth observation operators. Following the work of Lewis & Overton and Karmitsa, we investigate minimization of the data-assimilation cost functional using the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) quasi-Newton algorithm originally intended for smooth optimization and the limited-memory bundle method(LMBM) algorithm specifically designed to address large-scale non-smooth minimization problems. Numerical results obtained for the MLEF method show that theLMBMalgorithm yields results superior to the L-BFGS method. Results for 4D-Var suggest that L-BFGS performs well when the non-smoothness is not extreme, but fails for nonsmooth functions with large Lipschitz constants. The LMBM method is found to be a suitable choice for large-scale non-smooth optimization, although additional work is needed to improve its numerical stability. Finally, the results and methodologies of 4D-Var and MLEF are compared and contrasted. Copyright (C) 2011 Royal Meteorological Society