Objective functions

The objective functions in hydrological modelling are optimised with respect to model parameters in order to obtain the best possible agreement between the observed and simulated hydrologic variables (usually flows). Table 1 lists the objective functions commonly applied for hydrologic model calibration and evaluation.

Some objective functions indicate systematic errors (under- and over-estimation) or dynamic errors (timing). For example, presence of bias indicates under- and over-estimation of flows or runoff volume. Various hydrographs may result in the same bias because this performance measure is insensitive to dynamics of the simulated response.

On the other hand, low coefficient of correlation indicates only dynamic errors; therefore it could take a maximum value even if the simulated flows were negative because it does not recognise bias (Criss and Winston 2008). However, majority of the objective functions reflect both types of error (Krause et al., 2005). For example, Gupta et al. (2009) separated the Nash-Sutcliffe efficiency NSE in two parts: ratio between the mean simulated and observed flows, which indicates bias, and the correlation coefficient, which quantifies the dynamic error.

Moriasi et al. (2007) categorised the most frequently used objective functions into regression-based, dimensionless and error indices. The first group of objective functions is comprised of the correlation coefficient, linear regression slope and interception.

Dimensionless indices provide relative estimation of model efficiency and include e.g.

NSE, index of agreement d, etc. Error indices are based on the mean square error, MSE.

Criss and Winston (2008) analysed ability of several objective functions to capture errors in timing and proportional increase / decrease of a hydrograph. They suggested that some objective functions do not properly reflect these errors, and proposed the volume error VE.

It has been recognised that the objective functions based on squared residuals (such as RMSE or NSE) are sensitive to outliers. The values of such the objective functions are

sensitivity to high flows, NSE can be calculated from the logarithms of flows, square root of flows or their reciprocal values⁵ (Oudin et al., 2006; de Vos and Gupta 2010; Pokhrel et al., 2012; Seiller et al., 2012; Thirel et al., 2014). Lindstrom (1997) introduced a penalty to NSE in order to reduce NSE due to the runoff volume error. For balanced representation of systematic and dynamic errors in NSE, Gupta et al. (2009) proposed the KGE efficiency measure. Legates and McCabe (1999) suggested a general form of NSE, which enhances sensitivity to low flows. NSE can also be calculated for the flow duration curves. To cope with heteroskedasiticity in the residuals, Sorooshian et al. (1983) introduced HMLE.

The objective functions can be used as the evaluation criteria as well. This means that these functions are not included in model calibration, but they are employed to measure model performance instead. In addition to the objective function, Euser et al., (2013) proposed several “signatures” to test the realism of a hydrologic models, such as autocorrelation in the flow time series, rising limb density or peak distribution.

Further, Crochemore et al. (2015) studied the agreement between objective functions and expert judgement on model performance by conducting a survey among the hydrologic modellers. They revealed that the objective functions based on the squared or absolute error corroborate expert judgement about high flows. As for low flows, objective functions based on the log-transformed flows best reflect the expert judgment.

None of the objective functions is sufficiently versatile to reflect all aspects of agreement between simulated and observed flows. Model calibration should therefore employ several complementary performance criteria (e.g. Gupta et al., 1998; Moriasi et al., 2007).

Recommendations on the acceptable values of NSE and flow bias are presented by Moriasi et al. (2007).

5 Reciprocal values are calculated as (1/(Q+ε)), where ε is small constant (usually one per cent of mean flow value) to avoid dividing by zero (Thirel et al., 2014).

Table 1. An overview of the most frequently used objective functions for hydrologic model calibration against observed flows.

Objective function Equation Dimension Target value Comments and references

Relative flow bias

Bias which is not normalised is expressed in units of flow or runoff.

- 1 Insensitive to differences between simulated and observed flows (bias).

MAE lower than one half of standard deviation of the observed flows is considered low.

MAE is less sensitive to outliers than RMSE, therefore it is preferred over RMSE if outliers are present in the flow series (Legates & McCabe, 1999).

Root mean square

homoscedastic (Gupta et al., 1998; Romanowicz et al., 2013).

24

Table 1 (continued). An overview of the most frequently used objective functions for hydrologic model calibration against observed flows.

Objective function Equation Dimension Target value Comments and references

Transformed Root

Negative values indicate that mean value of the observed flows is better predictor than the model.

NSE is rather sensitive towards high flows due to square values of the differences.

NSE can be calculated using transformed flows (e.g.

log-transformed or reciprocal values of flows).

NSE can take low values if the observed flows exhibit small variability (Criss & Winston, 2008).

Linström measure LM

LM is obtained by modifying NSE to account for error in simulated runoff volume.

Value of w is commonly set to 0.1 (Lindstrom, 1997)

Kling-Gupta

KGE is obtained by balancing model performance in reproducing mean flows and flow variability and linear correlation between observed and simulated flows (Gupta et al., 2009).

Table 1 (continued). An overview of the most frequently used objective functions for hydrologic model calibration against observed flows.

Objective function Equation Dimension Target value Comments and references

Index of agreement d

- 1 Poor model performance may yield high values of this index (e.g. over 0.7) (Krause et al., 2005).

Volume error VE

VE denotes the flow volume common to the simulated and observed hydrograph and its complement denotes volume mismatch (Criss &

Winston, 2008).

HMLE is calculated from the flows that are transformed applyign Box-Cox transformation (Box

& Tiao, 1973).

λ is paramter of the Box-Cox transformation that has to be estimated in the calibration along with the free model paramters (Sorooshian et al. 1983).