On-line machine-learning forecast uncertainty estimation for sequential data assimilation

Quantifying forecast uncertainty is a key aspect of state-of-the-art numerical weather prediction and data assimilation systems. Ensemble-based data assimilation systems incorporate state-dependent uncertainty quantification based on multiple model integrations. However, this approach is demanding in terms of computations and development. In this work, a machine-learning method is presented based on convolutional neural networks that estimates the state-dependent forecast uncertainty represented by the forecast error covariance matrix using a single dynamical model integration. This is achieved by the use of a loss function that takes into account the fact that the forecast errors are heteroscedastic. The performance of this approach is examined within a hybrid data assimilation method that combines a Kalman-like analysis update and the machine-learning-based estimation of a state-dependent forecast error covariance matrix. Observing system simulation experiments are conducted using the Lorenz'96 model as a proof-of-concept. The promising results show that the machine-learning method is able to predict precise values of the forecast covariance matrix in relatively high-dimensional states. Moreover, the hybrid data assimilation method shows similar performance to the ensemble Kalman filter, outperforming it when the ensembles are relatively small. Artificial neural network training scheme. The training error is determined by the Frobenius norm between & sum;k$$ {\tilde{\boldsymbol{\Sigma}}}_k $$ and the training matrix & varepsilon;kf(& varepsilon;kf)T$$ {\boldsymbol{\epsilon}}_k<^>{\mathrm{f}}{\left({\boldsymbol{\epsilon}}_k<^>{\mathrm{f}}\right)}<^>{\mathrm{T}} $$, which is estimated from the approximated forecast error. image