PROMPT Implementation Phase 4 (PI-P4)

The goal of PI-P4 is to assess the potential effects of model biases found in PI-P2 and PI-P3 on the proposed policy options for ozone control. At this stage, all previous evaluation procedures should be completed. Therefore, what we expect as material for performing PI- P4 are (1) whether the model shows conceptually consistent behavior with observations, (2) whether the model can support various precursor controls in sub-domains, i.e., what types of control can be evaluated for effects given the model’s performance in these areas, and (3) how precisely the model can and must estimate control requirements. We recommend in PI- P4 that evaluators assess how the scientific biases found in previous procedures might potentially bias policy choices.

The difference between scientific bias assessment and science-policy bias assessment can be best illustrated with an example. As shown in Figure 2.5, suppose we have two sets of modeling input in which the only difference is the surface wind direction. If one wind set is perfectly matches observation and the other wind set does not match as well, the matched set is better scientifically than the other set. If the sources we want to control, however, are homogeneous around the receptor, the receptor becomes insensitive to the wind direction differences. Subsequently, the correctness of surface winds is not really important with respect to its value for decision making. In other words, both sets of wind can be used as inputs to PAQMs to test some policy options. This is an important aspect of the PI-P4 concept. PI-P4 should be a process for identifying model performance issues in terms of their importance with regard to science-policy questions. To achieve this aspect of PI-P4, evaluators and modelers need to communicate with policy makers in cataloging proposed control options as we discussed in the description of PI-P3.

Figure 2.5. Hypothetical sources (the grey area and the box ‘C’) and receptor (‘R’) in a PAQM. Suppose two meteorological inputs are provided, S winds (MET-A) and E winds (MET-B). If emission intensity from the grey area is homogeneous and other conditions are identical, except the wind directions as shown above, (a) the policy question about the effectiveness of control of emissions from the grey area on the receptor ‘R’ can not be

different by MET-A and MET-B even if MET-A is identical to observation. (b) However, the same type of question for the effectiveness of control of emissions from ‘C’ will be answered very different by two meteorological cases. In fact, MET-B will lead policy makers to wrong control by show effects of control of source ‘C’ at the receptor ‘R’ if there is a coincidental agreement between observation and model prediction with MET-B.

Our recommendation is to focus on those monitors with high confidence (i.e., Category MH-R) and/or with moderate confidence (i.e., Category MH-A) with further diagnostics or the assessment of control option effects. At the same time, model behavior at monitors with low level of confidence (i.e., Category MH-U) should be further examined to see if we can make improvements of model performance at those locations. If we cannot improve model performance, we need to explain why it is so and to assess the effects of our inability to enhance model performance on posed policy questions. Another important step we recommend is to inspect locations where the future ozone concentrations are high. If the future ozone is likely happening at locations where the model show moderate confidence (i.e. Category MP-A) or low confidence (i.e. Category MP-U), the whole SIP modeling process should be reviewed seriously, including the modeling episode selection, because we may be dealing with the ozone problem with a model that can not provide reliable answers to our policy questions.

As one of the most important outcomes from the application of PROMPT, we recommend evaluators create a GIS map showing the assessed reliability of model

performance on the proposed policy options for each episodic day. As shown in Figure 2.6, evaluators need to summarize their performance evaluation results on maps with detail commentaries including what form their confidence on the model performance. All

information used for their evaluation should be accessible and subject to discussion with the use of well-maintained Hyper Text Markup Language (HTML) documents through World Wide Web (WWW) for stimulating this task effectively. HTML pages are especially useful for on-going SIP modeling and there are ample tools to convert HTML pages into

documentation is concerned later. Because SIP modeling is constrained by non-scientific conditions, such as SIP submission deadlines, policy makers may have to make policy decisions even with SIP modeling results in which they have low confidence. In this case, model evaluators should prepare recommendations about how to use partial useful modeling results. We recommend that evaluators clarify what kind of policies proposed by policy makers should be more “limited” or “constrained”. This process should be iterative and interactive between policies makers and modelers. Also, we argue (1) a SIP based on limited model performance should in the SIP commit to future studies or research to resolve the uncertainties or issues and (2) the commitment needs to be explicit in the SIP documentation. Regardless of how the SIP modeling turns out, we argue that modelers need to provide a vindication of the model results (Jeffries, 1995b) by asserting and defending that “no one knows how to do this modeling any better that was done at the time with available resources and scientific knowledge.”

Figure 2.6. Example outcome of PI-P4. Areas are color-coded by an evaluator’s confidence on a model performance. The red circles represent the peak ozone observed at monitor sites and values in legends for ozone are in ppb.