A variance-minimizing filter for large-scale applications

A truly variance-minimizing filter is introduced and its performance is demonstrated with the Korteweg DeVries (KdV) equation and with a multilayer quasigeostrophic model of the ocean area around South Africa. It is recalled that Kalman-like filters are not variance minimizing for nonlinear model dynamics and that four-dimensional variational data assimilation ( 4DVAR)-like methods relying on perfect model dynamics have difficulty with providing error estimates. The new method does not have these drawbacks. In fact, it combines advantages from both methods in that it does provide error estimates while automatically having balanced states after analysis, without extra computations. It is based on ensemble or Monte Carlo integrations to simulate the probability density of the model evolution. When observations are available, the so-called importance resampling algorithm is applied. From Bayes's theorem it follows that each ensemble member receives a new weight dependent on its "distance'' to the observations. Because the weights are strongly varying, a resampling of the ensemble is necessary. This resampling is done such that members with high weights are duplicated according to their weights, while low-weight members are largely ignored. In passing, it is noted that data assimilation is not an inverse problem by nature, although it can be formulated that way. Also, it is shown that the posterior variance can be larger than the prior if the usual Gaussian framework is set aside. However, in the examples presented here, the entropy of the probability densities is decreasing. The application to the ocean area around South Africa, governed by strongly nonlinear dynamics, shows that the method is working satisfactorily. The strong and weak points of the method are discussed and possible improvements are proposed.