Technical evaluation 1: Bayesian networks to support water demand management legislation
5.3 Modelling issues arising from development of the conceptual model for WDM legislation
5.3.1 Development of preparedness strategies
The use of a single time-step in the conceptual model implies that it might be used in real-time decision-making involving the collection of evidence on water availability indicators at regular time to signal a requirement for changes in water conservation policy. However, using the model in this way might be criticised as encouraging a crisis approach to water management which is not recommended in the Sofia case.
Development of the conceptual model demonstrates that, theoretically, Bns could be used in contingency planning. To use the conceptual model developed in Chapter 5 above for contingency planning would require further detail in the forecasting sub-model, modelling of water supply to other sectors, and further analysis of risk thresholds for hypothesis variables. The characteristic of Bns to reverse probabilities, which allows both top-down and bottom-up belief propagation, supports their application for both backcasting and forecasting studies.
5.3.2 Cross-sectoral planning
Developing of the conceptual model highlighted the need for an in depth understanding of the decision process and local context. Three interconnected
decisions and three organisational perspectives provided the basis for the conceptual model. The model effectively represents the causal relationships influencing a multi-organisational decision process and inter-dependency between the MoEW’s, MoE’s and SVs decisions Although each decision could, in practice, be modelled individually, modelling the decisions as a decision stream is interesting because it prompts the user to question the role of the three organisations (i.e. MoEW, MoE, and SV) in WDM implementation and raises questions about who pays for WDM.
An aim of the field work described in Chapters 4 & 5 was to determine whether the collaborative decision process described by practitioners during the knowledge elicitation could be modelled using Bns. The results indicate that the combination of chance, decision and utility nodes can be used to model decision processes that involve decisions made by multiple organisations.
5.3.3 Developing the evidence-base for WDM legislation
The data collection for the conceptual model demonstrates a number of ways in which Bn model development facilitates the development of an evidence-base and management of uncertainty for water savings and WDM costs.
Knowledge elicitation can be used but requires careful planning and evaluation.
Water demand needs to be disaggregated into separate components (e.g. see Figure 4.10 in Chapter 4) and potential water saving allocated accordingly, rather than using aggregate demands. Local reports provide a further potential source of data and, if not available, once demand has been disaggregated into components, reports from WDM programmes in other regions can be used to construct and populate CPTs.
Further options for forecasting water savings are presented in Chapter 6 using increasingly data intensive methods including: knowledge elicitation, survey data and household metered demand data, to forecast water saving potential at the neighbourhood scale.
Developing methods for eliciting and calculating utility functions in Influence Diagrams (IDs) was not addressed in the case study field work and is an area that will need to be addresses if Bayesian modelling is to be used to inform policy decisions. The issue of the measuring the economics of conserving water is discussed briefly in the following section.
5.3.4 Estimating utility functions in Influence Diagrams
Water conservation results in several potential socio-economic and environmental benefits that are experienced in different ways at different scales and at different moments in time. For example, it contributes to a community’s resilience to drought conditions but as Bruneau et al (2003) point out, quantifying the benefits associated with building resilience into social, economic, and environmental systems continues to elude economists.
The European Union assert that the ‘full costs’ of water should be considered when making water allocation decisions. The different components of full costs are shown in Figure 5.7 below. The incremental build up of full costs is described in Appendix Q.
Figure 5.7. General water costs and value definitions (after Rogers, 1998)
The difficulty in quantifying benefits from conserving water will have an affect on how Influence Diagrams can be applied to support WDM implementation. Evaluation of the use of IDs and testing of different methods for eliciting utilities are identified as areas for future work in Chapter 9.
The following section discusses how Bns addressed structural uncertainties in modelling WDM legislation in the Upper Iskar.
5.3.5 Addressing structural uncertainties in the planning process
Structural uncertainties encountered during development of the conceptual model for the legislation stage arose from the initial lack of knowledge about uncertainties in cause-effect relationships and the mechanisms by which policy mechanisms impact
Environmental externalities
(e.g. higher value of alternative uses)
on these. Each cause-effect relationship required analysis of uncertainty so as to determine its candidacy for inclusions in the conceptual model.
The information flow requirements in Bns, i.e. decisions must be linearly-ordered so that there must be a path that contains all decision, determined how the conceptual model of WDM legislation in Sofia was structured and presented in two ways
Limitations of modelling feedback loops with Bns raises constraints in using them for forecasting the impact of different WDM programmes on future reservoir levels.
Knowledge elicitation and analytical approaches more suitable for hydrological modelling (e.g. System Dynamics) may provide a solution to support the development conditional probability tables and more detailed forecasts.
Secondly, the requirement to linearly-order decision nodes means that utilities for each separate decision are aggregated into the next decision node and the result can only be evaluated by the user once all the decision nodes have been instantiated.
This characteristic of Bns may constrain the applicability of Bns in some cases, for example, if the decision involves feedback cycles. For the decision process represented in the conceptual model in Chapter 5, however, a sequential modelling approach using chance, decision and utility nodes, appears to work well and successfully captures the interconnected nature of the three decisions being addressed as well as the uncertainty and risk inherent in the indicator variables (e.g.
metered demand, reservoir volumes, pay-back period). The utilities are determined by the reservoir level forecast (security of water supplies) and the cost and benefits of the WDM from the perspective of the water company.
Deciding on the position of arrows and their direction in a Bayesian network depends upon information flows, not physical flows. Achieving a model structure that conformed to the rules of information flows to achieve the logical cause-effect relationships required for the Sofia case involved numerous iteration and versions.
The value of the conceptual model as an artefact of the decision process can be considered as a viable output of the research that supports dissemination of knowledge. Examples of how the conceptual model could support structuring and prioritisation of data collection for WDM legislation in other river basin are given below.
5.3.6 Valuing the Bns as a dissemination tool
Firstly, development of the conceptual model identified the discretionary (outdoor summer) demand to total demand ratio as a useful and accessible indicator of (i) the feasibility of using seasonal pricing as part of a WDM programme (ii) the capacity for reducing water demand in the short-term using other measures to reduce outdoor use.
Secondly, to forecast water savings and their impacts on the water utility revenues, domestic demand requires disaggregation into different components (e.g. in the Sofia case these were: unaccounted for Water (UfW) and metered demand). These two components may be subject to further disaggregation, as shown in Figure 4.10 in Chapter 4.
A third transferable lesson is that, from the perspective of the water company, the payback period for WDM measures will always be determined by its variable operating costs which are composed of: energy costs, chemical costs, and raw water costs. The ratio of these components will vary for different regions. For example, the city of Sofia receives most (80%) of its water from the surrounding mountains so the Sofia water supply network is mainly gravity-fed, and this is the cause of very low energy costs. However, water companies who have large groundwater resources will have proportionally higher energy costs because pumping groundwater is more energy intensive than a gravity-fed system. Such regional characteristics change the proportion of variable operating cost components and, therefore, affect the payback period and economic feasibility of WDM measures.
The demonstration of the use of Bayesian networks in Chapters 4 & 5 provides evidence that Bns are suitable as a tool for recording examples of approaches to water management. Evaluation of the perceived effectiveness of Bns as a communication tool is reported in Chapter 7. If Bns can be applied effectively to communicate water demand management issues to a wider policy audience then it will provide evidence for their candidacy as a tool for disseminating knowledge to support WDM implementation between river basins. If used in this way they would provide a valuable secondary resource to facilitate the process of change required to achieve the demand-side ambitions of IWRM.
5.4 Conclusions
Development of the conceptual model in Chapter 5 provided evidence to examine three research questions, and the results are reported below in terms of strengths and weaknesses of the Bayesian approach.
Research question 3: How does Bayesian network modelling provide support for developing preparedness strategies?
For development of preparedness strategies, a strength of Bns demonstrated in Chapter 5, Figure 5.2, is the use of forward and backward propagation of conditional probabilities. This allows Bn models to be used to support both forecasting and backcasting studies. However, to avoid misunderstanding or discussions becoming unfocussed, the objective of the model needs to be clearly stated during the early stages of model development.
Once the network has been constructed, model instantiation makes it possible to quickly evaluate the impact of a range of future scenarios. This, along with their visual representation, which makes it easy for the user to gain a quick understanding of how the system works, makes Bns a potentially valuable too for supporting development of preparedness strategies.
Weaknesses of using the Bn approach for supporting preparedness strategies identified from model development are that although modelling over time-steps is possible with Bns, it increases model complexity. If the length of a time-step needs to be changed, all cpts in the model need to re-specified, which can be very time-consuming, and former research (Jensen, 2001) recommends that for modelling over multiple time-steps, the Bn model for each time-step should only include a minimum number of nodes (e.g. 3-5).
Research question 4: How does Bayesian network modelling provide support for decisions involving multiple organisations?
Bayesian modelling, and specifically Influence Diagrams (IDs), were demonstrated to provide potentially useful characteristics for supporting decisions involving multiple organisations. The ID in Figure 5.2 effectively represents the causal relationships and inter-dependency in a multi-organisational decision process involving three interconnected decisions. The sequential structure of IDs together with a suitable model instantiation procedure allows the user to see how each policy mechanism
effectively determines who pays for demand reduction. In addition, the visual representation in Bns makes it easy to demonstrate how a system functions.
Weaknesses of using Bns for decisions involving more than one organisation include the complexity of modelling over more than one time-step already mentioned above.
Also, finding sufficient data to quantify links between different disciplines, for example, when trying to place an economic value on water availability for human needs and in the environment, can be constraining when constructing a large Bn.
However, this is a universal problem for all interdisciplinary approaches, and can be helped by networks being well documented.
Research question 5: How does Bayesian network modelling address issues of structural uncertainty in the planning process?
Using Bayesian networks, it is easy to demonstrate the way in which a system functions through the use of nodes and directed links. This is relevant not only to physical flows, as demonstrated in the water balance model in Figure 4.1, but also to information flow as demonstrated in the conceptual model in Figure 5.2. The Bn model in Figure 5.2 is valuable as an artefact of the WDM implementation process. It is a viable output of the research that supports dissemination of knowledge about indicators and cause-effect relationships between them, to guide implementation of demand management strategies in other river basins. Once populated, parameter sensitivity analysis allows each cause-effect relationship in a prior model to be analysed for uncertainty so as to determine its candidacy for inclusions in the final model.
A weakness identified relating to research question 5 is that in large networks there is a danger of having too much information to take in and an instantiation procedure is therefore required in order to avoid subsequent analysis becoming unfocussed.
5.4.1 Recommendations for water demand management in the Upper Iskar
Construction of the conceptual model answered questions about the required timing of WDM implementation in Sofia. Data collected to support construction of the conceptual model indicated that a long-term planning approach to WDM is advisable in the Upper Iskar and Sofia case because measures such as seasonal (conservation) pricing and outdoor restrictions, which are usually used to achieve short-term savings, will have little impact on total demand in the Sofia case.
Although it was not possible to elicit conditional probabilities and utilities for the LAC nodes with informed practitioners within the time constraints of the case study field work, initial analysis revealed that the pay-back period for different options will be relatively long (i.e. greater than 25 years) and that the most cost-effective options (i.e. those with a shorter payback period) that should be considered in the first instance are pressure reduction and repair of the existing network, to reduce unaccounted for water (UfW).
Conclusions from constructing the conceptual model indicate that a risk management approach, involving long-term WDM measures with low-pay-back periods such as pressure reduction, repair of faulty pipes, and regulatory measures including a reduction in new-build design norms, should be introduced immediately in Sofia.
Introducing an efficiency standard on household water appliances to improve coverage of water saving technology is a further option that should be considered for immediate introduction and evidence was collected during a household survey to model the potential impact of such a policy and is presented in Chapter 6.
During the knowledge elicitation reported in Chapter 3 the need to develop an evidence-base for WDM programme design was identified as a constraint to achieving commitment to full-scale demand-side management. The constraint arises from the uncertainty of impacts of WDM measures that raises risks as to the feasibility of making investments in WDM. Furthermore, pilot studies are required and these can be costly in themselves. The use of Bns in addressing this and other issues relating to uncertainty about programme impacts at the design stage is examined in Chapter 6.