Assessing The Skill And Value Of Seasonal Climate Predictions

The main driver for the large research effort devoted to developing and improving seasonal climate prediction models is the fact that El Nino Southern Oscillation (ENSO) events (quasi-periodic fluctuations in Indo-Pacific Ocean sea surface temperatures and mean sea level pressure) represent, on a global scale, the greatest source of interannual climate variability and are, to some extent, predictable. Australia is considerably impacted by these fluctuations and although is served by several operational prediction schemes, the associated degree of skill is, at best, only moderate. There now exist quite a number of dynamically-based seasonal prediction models which are global in extent and there is considerable interest in developing methods for maximizing and quantifying their skill and utility to potential end-users. It is also possible to assess their performance by accessing hindcast (retrospective prediction) data. One of these models was developed by CSIRO and is based on the CSIRO Mk3 global coupled climate model. Results from seven other models which comprise the DEMETER ("Development of a European Multimodel Ensemble system for seasonal to in TER annual prediction") project were also assessed. This paper focuses on an assessment of the skill of the models at predicting rainfall for a catchment region of south-east Australia. In each case, rainfall hindcasts are compared with observed rainfall totals and also compared with observed inflows into one of the major reservoirs, the Burrinjuck dam. The major findings are: It is not possible to distinguish between the performance of the different models due to different sample sizes and periods for which hindcasts are available. Overall, the models exhibit an ability to capture, to some extent, variations in seasonal rainfall associated with ENSO events and this is evident in the fact that they exhibit skill in the extreme categories but not in the average category. The average success rate, while greater than that expected by chance or the strategy which assumes climatology, is not high and is expressed in the slight shifts in the probabilities for below average and above average tercile categories. As a rough guide, the model-based predictions provide an advantage over climatology 1 year in every 10. Taking into account the fact that rainfall and inflow predictions can be somewhat redundant when dealing with water storages, this may overestimate the potential utility to end-users. Finally, it has to be recognised that the economic value of predictions, no matter how skilful, can be diminished according to the costs/benefits associated with decisions made by the end-user. Assessing value is a more task which needs to be done on a case-by-case basis.