Regional air quality models and the regulation of atmospheric emissions

This paper investigates regional air quality model performance and the regulation of atmospheric emissions. Although evaluation of regional models cannot be reduced to a set of rules, the paper shows ways of developing better understanding of model performance. It draws on studies in recent years by the Environment Agency to quantify the uncertainty in predictions of regional air quality models. It is argued that a decision by a regulator on how to use a regional air quality model should be based on both operational evaluation (involving comparison with observation) and diagnostic evaluation (for developing understanding of the model), using operational and diagnostic metrics. Operational and diagnostic evaluations were undertaken, using a 'constructor' (CMAQ) and a 'seer' type (TRACK-ADMS) regional air quality model, for the secondary pollutants PM10, PM2.5 and ozone, though for episodic ozone it was not possible to define an appropriate performance metric. Neither type of model showed clearly better performance when applied to long-term average concentrations. There was not enough information to set a minimum margin of error in operational evaluations but margins of 20% or more are to be expected. Unlike operational metrics there is no obvious way of deriving diagnostic metrics. However a footprint diagnostic metric was shown to be a way to reveal the behaviour of PM10 and PM2.5 in both types of model. It is therefore suggested that seer models are used to reveal the structure of a model's underlying mathematical equations from which diagnostic metrics can be formed. In the absence of an objective basis for setting acceptance criteria for models, it is proposed that the underlying pragmatic principle should be to use whatever has comparable accuracy with the best existing international practice. For regulatory applications, the error expected in current types of air quality models should be a consideration in any decision made on the basis of models.