Generalised Likelihood Uncertainty Estimation

Generalised Likelihood Uncertainty Estimation (GLUE) is by far the most popular method of uncertainty analysis in hydrological modelling, and has been applied to numerous catchment scale models (e.g. Smith, 2011, McMichael et al., 2006, Cameron et al., 1999, Hossain et al., 2004). The GLUE methodology was developed by Beven and Binley (1992), and was inspired by Hornberger and Spear’s (1981) method of

sensitivity analysis (Vrugt et al., 2009b). GLUE methodology aims to address the issue of “equifinality” in models. The equifinality concept originates from the notion that there can be no single correct or optimal model. Equifinality describes how different sets of model parameters may lead to an equally good model performance. A simple illustration of this would be to take a simple linear equation: a + b + c = d. If we had an observation of the value d that was 9, there are many possible combinations of a, b and c that could provide that answer. Using integers alone (0-9), there are 55 possible combinations that would result in the answer 9. In hydrology modelling, the same issue applies. Different sets of values may lead to similar model outputs, and using a Monte Carlo sample, one would expect to see both good and bad

model outputs across a wide range of values for each model parameter, depending on the values of other parameters. This means that the ‘goodness’ of a model does not depend upon individual parameters, but on the whole set of parameter values, and the interactions between the parameters. Given that the structure of the model is adequate,

unrealistic parameter combinations will lead to poor model results.

GLUE uses this theory to produce a set of ‘good’ models that are taken forward for use in model predictions and projections. GLUE uses prior distributions of parameter values to generate random sets of

parameters using Monte Carlo simulation. The results of the model runs are then compared to observed data using a likelihood measure to assess the acceptability of each model based on the residuals. A specific likelihood measure is not defined, but is left for the modeller to determine according to their requirements. Models that reach a certain threshold in the likelihood measure are defined as “behavioural” and those that don’t, “non-behavioural”. When the model is used for projections, the behavioural models all contribute to the distribution of the projection, and are weighted according to their likelihood measure (Beven, 2012). Thus, there are several moments that introduce

ranges and distributions; when defining a sampling strategy; when deciding upon a likelihood measure; and when determining the

conditions upon which a model is accepted as behavioural or rejected as non-behavioural (Beven, 2012).

There has been significant debate in the literature surrounding the GLUE methodology, which has focused on the fact that GLUE is not formally Bayesian and is rather subjective in its approach. There have been three central debates in the literature, between those that believe GLUE is a useful working methodology for assessing uncertainty, and those that prefer to use more formal probabilistic approaches (Vrugt et al., 2009b). The provoking papers in these debates were “On

undermining the science” (Beven, 2006b), “Hydrological forecasting uncertainty assessment: Incoherence of the GLUE methodology” (Mantovan and Todini, 2006) , and “Pursuing the method of multiple working hypotheses for hydrological modeling” (Clark et al., 2011). The “On undermining science” debate was initiated by Keith Beven (Beven, 2006b), who asked whether uncertainties in models are overestimated by GLUE or other uncertainty estimation techniques, whether showing the results of uncertainty analyses to users and stakeholders would undermine their confidence in science, and how uncertainties could be constrained in future to improve model results. He concluded that uncertainty analysis need not undermine science, but called for better evaluation of uncertainty in hydrological models. Several replies suggested that whilst uncertainty need not undermine science, the concept of uncertainty needs to be better defined, and methods of uncertainty analysis better developed (Todini and Mantovan, 2007, Hall et al., 2007). It was also suggested that

uncertainty is all too often an afterthought in model development (Hall et al., 2007) and that uncertainties need to be made explicit in

The “incoherence of GLUE” debate was sparked by Mantovan and Todini (2006), who challenged the use of “less formal likelihoods” which lose the learning properties of the Bayesian inferential approach. Beven et al (2007, 2008) maintained that GLUE is appropriate and coherent according the Bayes theorem in “special cases where the modeller is

prepared to make very strong assumptions about the nature of the

modelling errors”. This debate continued with further challenges by

Mantovan et al. (2007), and concluded with Beven et al. (2008)

demonstrating the flexibility of the GLUE approach in “non-ideal cases”. The more recent debate with Clarke et al. (2012, 2011, Beven et al., 2012) focussed on the superficial rejectionist nature of GLUE from a Bayesian perspective, and concluded with recognition of the need to continue improving the process of model development and evaluation.

It is clear from the extensive literature surrounding the GLUE

methodology, and the many applications of GLUE in hydrology models, as well as other earth systems models, that it is a very popular and flexible approach to model uncertainty evaluation. It is also clear however, from the many exchanges between Professor Beven and other hydrologists, that there are two schools of thought regarding the application of formal and informal Bayesian methods, therefore a few of the Bayesian approaches to model uncertainty assessment will be discussed.