PROMPT Implementation Phase 1 (PI-P1)

The goal of PI-P1 is to confirm that a model is potentially capable of describing a specific ozone problem dealt with in a SIP. If a model shows ozone behavior consistent with what a conceptual model describes, we can say that the model is at least usable for the SIP modeling, i.e. testing a model’s descriptive power. The required information for PI-P1 is an operational PAQM and a conceptual model. Note that a model’s precision is not really a main subject at this phase yet.

We recommend two core tasks for evaluating a model’s descriptive power. The first task includes documenting and reviewing details of the modeling system setup including grid structures and available observations including the nature of measurements such as

spatiotemporal resolution. Also, it is highly recommended to plan how to use each class of observation can be used in further performance analyses. The second task includes

comparing the model’s behavior with the conceptual model based on observations. Relevant model setup and inputs should be reviewed to ensure they are the best

available or to find if there is an alternative. It is not clear how to define the appropriateness of using the term ‘best’. Like our scientific activities, however, the claim of what is ‘best’ is a result of dialogical consensus, not necessarily a unanimous conclusion (Crawford-Brown, 2005). We do not propose that every choice of the model setup should be right at this point because it is impossible to examine whether it is ‘right’ with limited information, which is almost always true to SIP modeling. Rather, we mandate to record what the rationales are for picking a specific set of model configurations so that we can check their appropriateness whenever we suspect part of model configuration causes performance problems. For inputs,

additional ones given resources. Again, the term ‘best’ is much more a practical term and is not about being accurate or correct scientifically.

One of the most important things in examination of inputs is (1) to check how raw outputs of meteorological models and emission models were converted and manipulated for simulations, if any, and (2) to prescribe and justify how to process those data in consistent resolution to observational data sets made by other than ground monitors. Any adjustments such as interpolating wind fields should be documented with justification, tools, and

instruction of how to do adjustments to ensure reproducibility. Also, the location information should be compiled precisely, especially with respects to monitor locations, along with detailed information on what geographical coordinate systems were used. Moreover, the resolution of observational data needs to be considered for the purpose of proper prediction-observation comparison to achieve the data quality consistency in MPE. It seems to be trivial at a glance but it is very critical when we do comparison of observation with prediction and try to figure out any root of discrepancies in the context of geographical relations among receptors and sources. We may conclude a model passes this part of the test if anyone who has a reasonable level of technical skills to operate PAQMs can reproduce the model simulations under the evaluation process with the provided documentation and inputs including raw meteorological models.

A conceptual model is a description of our understanding of the ozone problem inferred from all available observations and current theories of ozone formation and transport. Unfortunately, even though EPA guidance requires a conceptual model for a SIP modeling, the regulatory air quality modeling community does not have a formal framework for

meteorologists, inventory staffs, atmospheric scientists, field scientists/engineers, and stakeholder groups, if possible. Note that modelers are not necessarily in charge of developing the perceptual/conceptual model. If there is no conceptual model, it will be almost impossible to proceed further even though eventually the conceptual model can be revised depending on evaluation results and other new findings during a SIP modeling. Therefore, we simply compel modelers to acquire the conceptual model information or to request it from the groups in charge. This is a very important task that is often overlooked, under-funded, and performed late in most SIP works.

The main outcomes of PI-P1 are primarily identifying characteristics of ozone signals produced by a model and comparing them with the conceptual model’s description.

Description of characteristics should have temporal and spatial components such as when an ozone peak occurs at a monitor and where the monitor is located at in the study domain. The rate of ozone concentration change and dominant wind pattern are also desirable components of ozone characteristics description. Also, evaluators need to obtain information with regard to precision and accuracy of measurements because measurements are the ultimate material used throughout the MPE process. Unless specific information is provided, evaluators may use federal guidelines for referencing accuracies and precisions of measurements such as ‘Meteorological Monitoring Guidance for Regulatory Modeling Applications’ (US EPA, 2000) for ground monitors.

Even if a model may be able to pass some tests, especially statistical tests, any apparent anomalies should be marked for further examination. Evaluators may ask modelers (1) to identify and explain any illustrated weakness, and (2) to carry this list forward to be addressed in subsequent evaluation phases. If necessary, evaluators need to request

alternative episodes, improvement of meteorology, etc. Visualizing peak ozone for each day after sorting the monitoring sites by direction such as east to west is a simple graphical test that may help evaluators detect obvious geographical ozone biases such as shown in Figure 2.3. Depending on the timeline of the SIP submission and available resources and unless it is very clear that the model is not usable, evaluators may need to proceed to the next phase of MPE while keeping in mind the apparent weakness of the model.

Figure 2.3. An example bar chart showing daily peak ozone. Data used for this chart is from Houston-Galveston Mid-Course Review 1993 modeling case and all monitors are sorted by its location from west to east with observed ozone and model predicted ozone. As shown in the figure, there are large spatial discrepancies in peak ozone in model and real world at monitor sites. Even with these differences, this specific modeling case could pass EPA’s three statistical tests.