A spatial verification method applied to the evaluation of high-resolution ensemble forecasts

The verification of ensemble systems is being operationally carried out in several meteorological centres. However, the main operational ensemble systems have a coarser spatial resolution with respect to the deterministic runs. Only recently, high-resolution limited-area ensembles have started to be run on a regular basis. Their verification requires combining the usual probabilistic evaluation with the statistical verification techniques which are being developed for high-resolution model forecasts (1 - 10 km). These techniques permit to evaluate a deterministic forecast in a probabilistic manner, by taking into account the spatio-temporal distribution of the forecast at different scales. In this work, a spatial verification technique, called 'distributional method', is used to verify the Consortium for Small-scale MOdeling Limited-area Ensemble Prediction System (COSMO - LEPS) ensemble system, a mesoscale ensemble with 10 km horizontal resolution. The system is mainly designed to give probabilistic assistance in the forecast of severe weather, in particular of intense precipitation possibly leading to floods, hence verification is focused on the ability of the system in forecasting precipitation at high spatial resolution. The methodology is based on a comparison of forecasts and observations in terms of some parameters of their distributions, evaluated after the values are aggregated over boxes of selected size. In particular, performances in terms of average, a few percentiles and maximum forecast value in a box are considered. The system is compared against European Centre for Medium-Range Weather Forecasts Ensemble Prediction System (ECMWF EPS), addressing the issues of an intercomparison between a higher-resolution smaller-size ensemble and a lower-resolution larger-size one. Results show that when the forecast of the average amount of precipitation over an area is concerned, COSMO-LEPS is more skilful than the Ensemble Prediction System (EPS) only from the resolution point of view. Therefore, although not properly calibrated, it is more capable of distinguishing between events and non-events, especially for moderate and high precipitation. Furthermore, COSMO-LEPS has skill in forecasting the occurrence of precipitation peaks over an area, irrespective of the exact location. The analysis of the score behaviour as a function of the distribution parameter shows that EPS has the maximum skill in reproducing the central part of the observed precipitation distribution over an area of about 10000 km(2), while COSMO-LEPS is more skilful in reproducing the tail of the observed precipitation distribution. The problem of the predictability of precipitation at different spatial scales is also investigated, showing the role of different system resolutions. Copyright (c) 2008 Royal Meteorological Society.