Quality Assimilation and Validation Process For the Ensemble of Environmental Services

(1)

Quality assurance plan for the Ensemble

air quality re

Date:

07/2014

Authors:

Laurence

ROUÏL

(INERIS),

Reference

: D112.3

Quality assurance plan for the Ensemble

air quality re-analysis chain

Laurence

ROUÏL

(INERIS),

Quality assurance plan for the Ensemble

analysis chain

(2)

File: MACCII_EVA_D112.3/.pdf

Date

07/2014

Status

Final Version

Authors

Reference

Laurence ROUÏL

D112.3

Use and reproduction of this report or pa

Appropriate non-commercial use will normally be granted under the condition

that reference is made to MACC

Please enquire with: info@gmes

This document has been produced in the context of the MACC

Composition and Climate - Interim Implementation). The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7 THEME [SPA.2011.1.5 under grant agreement n° 283576. All information

warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commissi

respect of this document, which is merely representing the authors view.

/.pdf

2

Final Version

Laurence ROUÏL

(INERIS)

Use and reproduction of this report or parts of it may be restricted.

commercial use will normally be granted under the condition

that reference is made to MACC-II.

[email protected]

This document has been produced in the context of the MACC-II project (Monitoring At

Interim Implementation). The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7 THEME [SPA.2011.1.5 under grant agreement n° 283576. All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commissi

respect of this document, which is merely representing the authors view.

rts of it may be restricted.

commercial use will normally be granted under the condition

II project (Monitoring Atmospheric Interim Implementation). The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7 THEME [SPA.2011.1.5-02]) in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission has no liability in

(3)

₃

Table of content

1. Executive summary ... 6

2. The EVA production chain ... 6

3. Input data for the EVA ensemble system ... 8

(4)

₄

Glossary

AIRBASE European Air Quality database (

http://air-climate.eionet.europa.eu/databases/airbase/)

Analyses Maps of air pollutant concentrations fields issued from numerical model results combined with up-to-date available observation data to improve their accuracy in the vicinity of measurement points. In MACC-II, they are produced routinely on a daily basis.

AOT 40 Accumulated Ozone over the 40 ppb Threshold

AQD Air Quality Directive

Assessments Quantitative evaluation of air quality fields based on validated data and numerical model results

CERFACS Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique (France)

Data assimilation Mathematical process to incorporate observations in a numerical model of physical systems

EDA Air Quality data assimilation sub-project in the MACC and MACC-II projects

EEA European Environment Agency

ENS Air quality forecasting and analysis sub-project in the MACC and MACC-II projects

Ensemble Model Combination of various results from various models. This can be a simple average (median), or a weighted average resulting from analysis of models’ behaviour over past periods. The models building-up the ensemble can correspond to different systems (multi-model approach) or to the same modelling system fed with different input datasets.

EVA: Air quality validated assessments sub-project in the MACC and MACC-II projects

FMI Finnish meteorological Institute

KNMI Royal Netherlands Meteorological Institute

LISA Laboratoire Interuniversitaire des Systèmes Atmosphérique (France) Météo France French Weather Services

Met.no Norwegian Meteorological institute (Norway)

Raw model data Model results directly issued from the modelling chain, without any post-treatment process

Re-analyses Maps of air pollutant concentrations fields issued from numerical model results combined with validated observation data to improve their accuracy in the vicinity of measurement points

RIUKK Rhenish Institute for Environmental research at the University of Cologne (Germany)

RMSE Root Mean Square Error. It gives the standard deviation of the model prediction error. A smaller value indicates better model performance. SMHI Swedish Meteorological and Hydrological Institute (S)

(5)

₅

SOMO35 Ozone concentrations accumulated dose over a threshold of 35 ppb TNO Netherlands Organisation for applied Scientific Research (NL)

VOC Volatile Organic Compound

WHO World Health Organization

(6)

₆

1. Executive summary

This note describes the process for the production of European Air quality Validated Assessment reports. Those reports result from the so-called “EVA regional air quality service” developed within the MACC and MACC-II projects. The first AQ assessment report was published in 2010 and was related to the year 2007. The last one, published in summer 2014 relates to the year 2012. During those five years 6 validated assessment reports have been published to describe air quality in Europe. The process is now well-defined and tested, and five years of development allowed fixing a number of operational issues

With this experience we are now able to describe a robust and straightforward process for the production of Ensemble air quality re-analyses throughout the Europe, based on data assimilated model results provided by seven European chemistry transport models.

Because the aim of this service is to comply with policy makers needs and to support decision, we have to guarantee a certain level of service (in terms of availability of data) and quality and reliability of the products (reports, maps, graphs) and data (numerical data generated by the models). The process defined to commit ourselves with those objective is described in the present document.

2. The EVA production chain

EVA service is based on two levels of re-analyses of air pollutant concentrations resulting from the combination of chemistry-transport models’ results and validated observations. Seven chemistry transport models are implemented to run this services, the same that are run to provide daily forecasts and air quality analyses for the ENS service. The same model configurations should be used.

Re-analyses are data assimilated air pollution fields. Data assimilation processes for various types of data (in-situ regulatory data, in-situ research data, Earth observations) are developed and evaluated within the EDA service.

ENS, EDA and EVA form a consistent ensemble of tools and products to provide European air quality community with relevant, accurate, reliable and consistent information.

Actually consistency of tools and data is one of the main characteristic and added-value of the MACC-II services.

Interlinkages of the EVA services with other MACC-II services are illustrated by Figure 1. EVA benefits from all QA/QC efforts made in the other services:

- Versioning and traceability of the evolution of the individual regional chemistry-transport models is followed in the ENS model dossiers,

- Validation of the data assimilation methods is ensured by the EDA services

- All models use the same input data provided by other mACC-II services: emissions, fires, global production.

(7)

Figure 1.Interlinkages of the EVA services (green central box) with other MACC

EVA service is based on two levels of production:

- Decentralised production is under the responsibility of the modelling teams. The models are run on computing

responsible for the quality of the production (yearly re

the time line (driven by the date of the publication of the EVA report), and with the quality of the model results.

- Centralised production is ensured by the leader of the service, INERIS. It consists in the compilation of all model results,

to calculate an “ensemble”, verification of individual and ensemble model results, publication of the assessment and the validation report.

Both levels of EVA production are synthesized in /.pdf

7

Interlinkages of the EVA services (green central box) with other MACC

EVA service is based on two levels of production:

Decentralised production is under the responsibility of the modelling teams. The models are run on computing systems hosted by the modelling teams. They are responsible for the quality of the production (yearly re-analyses) with respect with the time line (driven by the date of the publication of the EVA report), and with the quality of the model results.

roduction is ensured by the leader of the service, INERIS. It consists in the compilation of all model results, interpolation of these results on the same grid to calculate an “ensemble”, verification of individual and ensemble model results,

f the assessment and the validation report. Both levels of EVA production are synthesized in Figure 2.

Interlinkages of the EVA services (green central box) with other MACC-II services

Decentralised production is under the responsibility of the modelling teams. The s hosted by the modelling teams. They are analyses) with respect with the time line (driven by the date of the publication of the EVA report), and with the roduction is ensured by the leader of the service, INERIS. It consists in interpolation of these results on the same grid to calculate an “ensemble”, verification of individual and ensemble model results,

(8)

Figure 2.

3. Input data for the EVA ensemble

A very important point is that all models share the same input data: - Meteorological forcings come from ECMWH re

- Emission inventory is developed and maintained by the service dedicated to emissions

- Fire emissions are provided by the service dedicated to the comp pollutants emitted by forest fires

- Boundary conditions are global

It means that variability in model responses reflects the inherent uncertainty of model parametrisations, which are different f

This is very important when the Ensemble is built up to limit sources of divergence. Therefore local modelling teams

services.

For observations assimilated in the data

- At least the modelling teams have to use AIRBASE datasets

INERIS : one set is dedicated to data assimilation and the other one to evaluation and both do not overlap. INERIS defined rules to split th

for data assimilation, which should be representative of the model resolution, and evenly distributed over the domain. Subsets are reviewed each year to account for changes in AIRBASE from a year to another (stations added or

operational rate ...); /.pdf

8

Figure 2.Decentralised and centralised production in the EVA service

Input data for the EVA ensemble system

t is that all models share the same input data: Meteorological forcings come from ECMWH re-analysis system

Emission inventory is developed and maintained by the service dedicated to Fire emissions are provided by the service dedicated to the comp

pollutants emitted by forest fires

Boundary conditions are global-re-analyses elaborated by the global services.

It means that variability in model responses reflects the inherent uncertainty of model parametrisations, which are different from a model to another.

This is very important when the Ensemble is built up to limit sources of divergence. Therefore local modelling teams have to use the same input data

For observations assimilated in the data-assimilation systems there is more flexibility: At least the modelling teams have to use AIRBASE datasets split in two subsets

one set is dedicated to data assimilation and the other one to evaluation and INERIS defined rules to split these subsets: more observations for data assimilation, which should be representative of the model resolution, and evenly distributed over the domain. Subsets are reviewed each year to account for changes in AIRBASE from a year to another (stations added or removed, insufficient AIRBASE subsets are distributed by INERIS to the modelling

Decentralised and centralised production in the EVA service

Emission inventory is developed and maintained by the service dedicated to Fire emissions are provided by the service dedicated to the computations of air

analyses elaborated by the global services.

It means that variability in model responses reflects the inherent uncertainty of model This is very important when the Ensemble is built up to limit sources of divergence. from other MACC

ystems there is more flexibility: split in two subsets by one set is dedicated to data assimilation and the other one to evaluation and ese subsets: more observations for data assimilation, which should be representative of the model resolution, and evenly distributed over the domain. Subsets are reviewed each year to account for removed, insufficient AIRBASE subsets are distributed by INERIS to the modelling

(9)

₉

teams when they are released by the EEA, generally in February of the year Y + 1 for the year Y.

- Other in-situ data can be used, for instance when research networks provide relevant information, for instance to study an episode. Use of research in-situ data for assimilation in the models have not been encountered over the past 5 years. Such data are rather used for evaluation or for complementing the analysis of an episode

- Satellite data are available but the data assimilation chains have not the same maturity to use them. For instance only EURAD and LOTOS-EUROS assimilate operationally satellite observation of NO2. Choice is free for the modelling team and

generally Earth observations are not use for the validation process. So there is no overlap.

Considering the chemistry-transport models, all of them are described in detail, with their evolution, in the dossiers maintained by the ENS service. The note D112.2 describes quality assurance plans adopted by the individual models to guarantee the quality of their production. A summary of their status is given in the table below which is updated and provided in annex to the validated assessment reports.

INERIS as the coordinator of the service propose output formats for the individual model results which are base on the netcdf format (see next section).

Model DA process Pollutants

concerned

Data sources Operational production

CHIMERE Optimal interpolation : kriging observation data with CHIMERE as external drift Ensemble Kalman filter O3, PM10 O3 AIRBASE AIRBASE

Under evaluation : O3 partial tropospheric columns (IASI)

Yes

EMEP 3D-VAR NO2

O3, PM

OMI NO2 tropospheric

column , AIRBASE AIRBASE

AIRBASE

Yes

EURAD Intermittent 3DVAR O3, NO2,

NO, CO, SO2, PM10

AIRBASE in situ

measurements

MOSAIC air borne in situ measurements

NO2 tropospheric column

retrievals from OMI, GOME-2, SCIAMACHY

MOPITT CO profiles

(10)

File: MACCII_EVA_D112.3/.pdf ₁₀ LOTOS-EUROS Ensemble Kalman filter O3 NO2, PM10, , SO2, SO4 AIRBASE

AIRBASE and OMI NO2

tropospheric columns AIRBASE

Yes,

MATCH 3D-VAR with

transform into spectral space O3, NO2 PM10, PM2.5 AIRBASE AIRBASE AIRBASE Yes, MOCAGE 3D-VAR O3 NO2 PM10 AIRBASE AIRBASE AIRBASE Yes SILAM 3D-VAR 4D-VAR O3, NO2, SO2 AIRBASE AIRBASE Yes

4. Description of the ENSEMBLE operational production for EVA reports

A synthetic scheme (Figure 3) shows the general organization set-up all along the six first months of the year to produce EVA reports for year Y-2 based on the work of seven regional modeling teams. This is also supported by other MACCII activities like those performing by the global community and the emission community.

The first step of this production starts in February with the sending of email towards each regional team with enclosed document describing the input datasets available and their sources that should be used for the reanalysis runs. Additional information is also provided like the netcdf format that should be use to provide the model outputs on the INERIS ftp server1.

Later in March, once the last release of airbase (EEA) dataset occurs, INERIS launches several processing to split it into two datasets, one for the evaluation and the other for assimilation. Then a time period of two months is opened for model running. The ultimate deadline for output data provision on INERIS ftp server is fixed on mid-June.

From the files gathered on the INERIS ftp server, the total amount of data processed is around 500 gigabytes to put the model outputs on the same grid with an horizontal resolution of 0.1° x 0.1° over Europe. Then the production of the ensembles is established for the simulation and reanalysis with the median concentration.

The air quality indicators are assessed using the reanalysis ensemble (ENSa) and use to write the air quality assessment over Europe for year Y-2. Around 30 figures representing indicators for the four pollutants are produced among them the seasonal averaged concentrations, the number of threshold exceedances for O3 and PM10 regarding the regulatory standards …

1

Experience showed that it can be an important issue if the netcdf formats are not well defined. The netcdf format is well-suited to our needs but it offers a lot of flexibility to build up the files. So post-processing can become very complicated and source of errors.

(11)

All the model outputs (raw simulations and reanalysis) including Ensembles are evaluated using the observation dataset dedicated to evaluation through statistical skill scores (bias, RMSE and correlation). More than 200 figures are made available for the evaluation report.

Concerning QA/QC there are several important

- May : When the modelling teams send their files to INERIS they are scrutinised to check whether the file is complete, with the correct

to the model data provider;

- May-June: Then the content of the model runs are assessed using the validation observation datasets and usual statistical indicators are

coefficient and root means square error. information are drawn for each model. validation. Resubmission

- When the first version of the EVA report is ready by the end July it is sent to modelling teams for an ultimate checking.

we manage to implement a set /.pdf

11

uts (raw simulations and reanalysis) including Ensembles are evaluated using the observation dataset dedicated to evaluation through statistical skill scores (bias,

More than 200 figures are made available for the evaluation report.

Figure 3.:Time frame of the EVA report production

ng QA/QC there are several important steps:

the modelling teams send their files to INERIS they are scrutinised to whether the file is complete, with the correct format. If not a

to the model data provider;

n the content of the model runs are assessed using the validation and usual statistical indicators are calculated:

root means square error. Taylor diagrams that synthesise all skill are drawn for each model. The results id sent to the modelling team for esubmission is possible until mid-June.

When the first version of the EVA report is ready by the end July it is sent to modelling teams for an ultimate checking. In the future, this steps will be reduced if we manage to implement a set-up with an actual editorial committee.

uts (raw simulations and reanalysis) including Ensembles are evaluated using the observation dataset dedicated to evaluation through statistical skill scores (bias, More than 200 figures are made available for the evaluation report.

:Time frame of the EVA report production

the modelling teams send their files to INERIS they are scrutinised to not a message is sent n the content of the model runs are assessed using the validation calculated: bias, correlation diagrams that synthesise all skills results id sent to the modelling team for When the first version of the EVA report is ready by the end July it is sent to In the future, this steps will be reduced if up with an actual editorial committee.