Uncertainty Analysis

Different strategies and methods exist for uncertainty analyses. For quantita-tive data, especially for experimental results, it is common to describe the data by using data characteristics, such as mean and standard deviation. For expert judgements, qualitative scores can be used, for example by evaluating the confi-dence into the values as ‘high’, ‘medium’ or ‘low’. It should however be clearly stated what these terms mean in the context of the assessment, otherwise the evaluation can become a bit arbitrary. Alternatively, confidence can be evaluated by providing minimum, maximum and most likely estimates, which subsequently can be transferred into triangular distributions. Another possibility is to provide

confidence intervals, such as ‘x % confidence that the value of factor y is within range z’.

In case of a purely quantitative model (such as the LCC model), it is common to transfer the initially deterministic model into a probabilistic one. This can be done by modelling inputs with distribution functions, applying Monte Carlo analysis and evaluating the results statistically. In case of a qualitative or semi-quantitative model (such as for EI and RA), this may not be feasible, due to limited data and knowledge of the problem. Instead, it may be possible to elicit a qualitative judgement about the confidence in the given values.

3.4.2.1 Uncertainty Analysis of LCC model

The uncertainty analysis for the LCC model is done by transforming the originally deterministic model into a probabilistic one. Efforts are focussed on the highest contributing input factors, which have been identified through the sensitivity analysis. The approach is depicted in Figure 3.12.

Figure 3.12: Uncertainty analysis of LCC model

For an initial assessment, the inputs are modelled as triangular distributions. In future analyses, this could be refined by using other types of distributions. Trian-gular distributions have the advantage that they are simple and straightforward to define in case of limited data or expert judgement. Triangular distributions have only three input parameters: the minimum and maximum value, and an estimate for the most likely value, which defines the modal value of the triangle.

The width of the triangle reflects the associated variability and uncertainty.

The input distributions are determined based on refined literature research, ex-pert elicitation and market analysis. Subsequently, the Monte Carlo simulation

is executed as follows: Random values are generated and applied to the inverse Cumulative Distribution Function (CDF), in order to generate a number of ran-dom input variables that are distributed according to the chosen probability dis-tribution function. An estimate about the minimum number n_min of required Monte Carlo runs for a required confidence level L and an expected probability p of one independent variable can be determined with Equation 3.25 (Melchers 1999).

n_min = −ln(1 − L)

p (3.25)

In case of multiple independent variables the results have to be raised to the power of the number of independent variables, assuming that the functional relationship (linear, quadratic, etc.) is similar for all four independent variables.

Since the model is set up as a relative model, it is important to normalise the inputs so that the baseline always has a value of 100 %. The LCC results are calculated for each set of variable input parameters and the outputs are described statistically, e.g. through their mean and standard deviation, or alternatively by their quartiles. In this way, an attempt is made to develop a sound understanding about the most likely range of performance of the alternative options compared to the baseline, by factoring in the uncertainties associated with the highest contributing input parameters.

3.4.2.2 Uncertainty Evaluation of Risk end Environmental Impacts

Uncertainty evaluation for the EI and RA models differs from the quantitative approach for the LCC model, because they are based on semi-quantitative expert judgement. Rather than conducting fully quantitative uncertainty analyses, the focus therefore is on evaluating the confidence in the input values. In an ideal situation, confidence should be evaluated at every step of the evaluation. In the case of the methodology outlined above, this would mean to provide confidence values at the data collection stage, complemented with a confidence assessment for the evaluation of both the performance as well as the significance scores.

This approach may however not be very feasible for practical applications. In-stead, the uncertainty evaluation can be focussed on the aspect that is the key distinguishing factor between the different alternatives, namely the performance score P_i^j. Since this is provided through semi-quantitative assessment (as out-lined in Section 3.3.2), the confidence in the estimated values should be collected alongside. This can again be done through a qualitative judgement, for example ranging from ‘very high confidence’ to ‘very low confidence’, similar to the eval-uation of the significance. This evaleval-uation should however be based on a sound understanding of the uncertainties present in the preceding steps and the under-lying data. Therefore, wherever possible, this information should be collected alongside.

The evaluation of the uncertainty can be used in different ways: Either it can simply be displayed and communicated alongside the evaluation, or it can be used to target further research efforts effectively towards potential key issues that show high uncertainty and have a high contribution, as discussed above.

Additionally, it is possible to provide worst case and best case estimates, for example by varying the scores to an extend that is proportional with the degree of uncertainty. For example, if uncertainties are evaluated for the performance scores P_i^j, a variation of ∆P_i^j can be applied, which is proportional with the associated uncertainty.

3.5 Multi-Criteria Decision Analysis (MCDA)

This section provides an overview about the choice of MCDA, followed by a discussion about the weighting and aggregation scheme for the chosen MCDA technique. The details related to the case study are presented in Section 4.5.