On Some Limitations of Current Machine Learning Weather Prediction Models

Machine Learning (ML) is having a profound impact in the domain of Weather and Climate Prediction. A recent development in this area has been the emergence of fully data-driven ML prediction models which routinely claim superior performance to that of traditional physics-based models. We examine some aspects of the forecasts produced by three of the leading current ML models, Pangu-Weather, FourCastNet and GraphCast, with a focus on their fidelity and physical consistency. The main conclusion is that these ML models are not able to properly reproduce sub-synoptic and mesoscale weather phenomena and lack the fidelity and physical consistency of physics-based models and this has impacts on the interpretation of their forecasts and their perceived skill. Balancing forecast skill and physical realism will be an important consideration for future ML models. The last few years have seen the emergence of a new type of weather forecasting models completely based on ML technologies. These models do not codify the physical laws governing atmospheric dynamics but learn to produce forecasts from historical reanalysis data sets of the Earth system like the ECMWF ERA5. In this work we show that the forecasts produced by some of the leading ML models are physically inconsistent and should be better considered as post-processing algorithms rather than realistic simulators of the atmosphere. The challenge for next generation of ML models for weather forecasting will be to improve their fidelity while maintaining forecast skill. Forecasts from Machine Learning (ML) models have energy spectra notably different from those of their training reanalysis fields and Numerical Weather Prediction models This results in overly smooth predictions and weather phenomena at spatial scales shorter than 300-400 km are not properly represented Fundamental physical balances and derived quantities are not realistically represented in the forecasts of the ML models