Evaluation of the regulatory model

The regulatory model is the combination of the computer model with the environmental scenarios and the model parameters. Figure 5 in EFSA PPR Panel (2014) has been adapted to show the relevant parts of the general regulatory model that apply to TKTD models (Figure 38).

7.6.1.

Evaluation of the environmental scenarios

One of the benefits of TKTD modelling is that all the data (simulated time series instead of maximum or average concentrations) from the predicted exposure profiles (e.g. from the various FOCUS scenario) can be used as input for simulations.

For GUTS models using FOCUS scenarios as input, no further definition of the environmental conditions is needed, since the model parameters will be based on data obtained from experiments performed under standard laboratory conditions, and pesticide concentrations will have been generated using the relevant FOCUS scenarios, which consider factors such as soil type, rainfall and agronomic practice.

For DEBtox models, a basic environmental scenario has to be defined including temperature and food conditions. The physiological part of the model has the potential to extrapolate predictions of growth and reproduction to variable environmental conditions, in terms of e.g. temperature and food availability. Basically, accounting for variable energy intake is one of the main ideas behind DEB modelling. The application of the DEB theory for the prediction of toxicological effects, however, requires environmental scenarios to be fixed to the laboratory conditions of the experiments used for

calibrating the TKTD part of the model. This appears necessary, because it is not possible to assess whether the extrapolation of toxicity to other environmental conditions would work. In case appropriate validation data for variable temperature or food conditions is provided, explicit extrapolation to variable environmental conditions and hence the definition of different environmental scenarios could be possible.

It is considered appropriate to fix abiotic factors of the environmental scenarios to the laboratory conditions of the experiments used for calibrating the TKTD part of the model, and also to ignore ecological factors such as species interactions. This is because the modelling will be used with the equivalent of Tier-1 or Tier-2 Assessment Factors, which should already cover for the extrapolation from laboratory tofield conditions.

For primary producer models, a basic environmental scenario has to be defined including temperature and/or light conditions, and nutrients availability. The conditions used in the model should mirror the conditions in the experiments used for calibration, since this is in line with Tier-2A and Tier-2B, where additional species or refined exposure scenarios are tested in laboratory experiments without also using realistic light conditions and nutrient levels.

7.6.2.

Evaluation of parameter estimation

Parameter estimation requires a suitable data set, the correct application of a parameter optimisation routine and the comprehensive documentation of methods and results. Supporting data for TKTD models have to be of suf_{ficient quality (Section}7.2), be relevant for the risk assessment and ful_{fil a set of basic criteria. One important aspect about the data requirements is to have experimental} observations of the relevant endpoints (e.g. mortality or reproduction) for a sufficient number of time- points in order to have an appropriate degree of freedom to calibrate the model. The number of observations can vary per TKTD model, suggestions are given in the respective checklists (Annex A, Annex B and Annex C). Another important aspect for the calibration data set is that the response in the observed endpoint should range from no effects up to strong effects, ideally from 0% to 100%. The model can only capture the quantitative relationship between exposure and effects when a clear effect is visible in the underlying data. Parameter estimates can show reduced accuracy and precision when the experiments do not show a clear and comprehensively covered dose–response pattern. It is Figure 38: Schematic representation of a TKTD regulatory model (orange boxes). The environmental scenario feeds into the exposure model in combination with information on pesticide properties and uses. The exposure model in turn delivers the exposure profile, which is used as input by the TKTD models. Altogether, the regulatory model delivers an output, i.e. the endpoint of the assessment. For DEBtox and primary producers’ models, as indicated by the dotted arrow and the associated box, the recommendation is, for the time being, to fix the abiotic parameters to the laboratory conditions of the experiment used for calibrating the model. If in the future the relationship between toxicity and environmental conditions will be better understood and described, such recommendation can be revised. The figure represents an adaptation of Figure 5 in EFSA PPR Panel (2014)

difficult to define a minimum required level of response in the data, but a maximum effect of less than 50% in acute assessment for effects on invertebrates and chronic effect on plants, and of less than 90% in chronic assessments for effects on aquatic invertebrates and fish, should be well justified. In addition, the time course of the experiment used for calibration should be adjusted to capture the full toxicity of the pesticide (attention should be paid to time needed for the onset of effects, occurrence of delayed effects, accumulated toxicity, etc.).

It is possible that more than one suitable data set for calibration of the model does exist, since all available reliable data sets (apart from those used for validation) should be used (even non-standard test data when available and considered reliable). In such a case, parameter optimisation can be done separately per data set, resulting in one optimal parameter set per data set. There are at least two strategies how to continue from there. One option is to predict the validation data set based on each parameter set and to select the parameter set that shows the best performance (see Section 7.7.2). The other option is to test whether the distributions of the parameter values are significantly different, to merge the corresponding data sets and to re-calibrate the parameters for the merged data set in case they are not.

Some TKTD model parameters may also be taken from the literature. If this is the case, it should be checked whether they are reasonable, and if uncertainty limits are also provided.

Model parameters are always estimated for a specific combination of species and compound (see Chapter 3 for background information). For the evaluation of parameter estimation, also called model calibration or fitting, the method used has to be documented in detail, including settings of optimisation routines, the numerical solver (including settings) that was used for solving the differential equations, and the method for the calculation of parameter confidence/credible limits. Knowledge about these settings is important to allow experts to further evaluate parameter estimation when there is uncertainty about the quality of the results. Detailed background information on parameter estimation is given in Section 4.1.3, and example documentations for parameter estimation process and results are given in Section 4.2.2.1and AppendixA.1and A.2.

The results of the parameter optimisation process are not limited to the optimal parameter set alone. The optimal parameter values must be given together with confidence or credible intervals to allow evaluation of their uncertainty. Values of the objective function for calibration (e.g. log-likelihood function) need to be documented with sufficient numerical precision to allow for independent checks of the optimisation process. For an evaluation of the model calibration, plots of the calibrated models in comparison with the calibration data over time should be available and the visual match should be reasonable. Additional goodness-of-fit criteria can be reported, at least a posterior predictive check should be available and documented (example in Figure 17).

One specific of the parameter estimation for GUTS is how the background mortality rate is considered in the optimisation. The standard way is to estimate all parameters simultaneously, including the background mortality in the experiments because control data may contain information about TKTD parameters; in addition, it is also possible to check for potential correlations between model parameters. Alternatively, the background mortality rate constant can be calibrated to the observed mortality in the controls and be fixed in the calibration of the other GUTS parameters to data from the treatments.

All aspects in this section are condensed into corresponding checklists in Annex A (GUTS), Annex B (DEBtox) and Annex C (primary producers).