Decreasing Climate-Induced Water Supply Risk Through Improved Municipal Water Demand Forecasting

(1)

Sectoral Applications Research Program Climate Program Office

Oceanic and Atmospheric Research National Oceanic and Atmospheric Administration

American Water Works Association Kearns & West

George Washington University University of Colorado-Boulder Hazen and Sawyer

July 2013

Decreasing Climate-Induced

Water Supply Risk Through Improved

(2)

i

Acknowledgements

We would like to acknowledge NOAA for providing the funding for this project and we would like to thank Nancy Beller-Simms from NOAA for her support. We would also like to thank the participants in the two project workshops and the five webinars for their time and expertise: Washington, DC, Workshop Denver, CO, Workshop

Alison Adams, Tampa Bay Water Sarah Deslauriers, Carollo

Veronica Blette, EPA Ben Dziegelewski, University of Illinois

Erica Brown, AMWA Rick Holmes, SNWA

Jim Chelius, American Water Pam Kenel, Black & Veatch Roger Cooke, Resources for the Future Alfredo Rodriguez, Aurora Water

Pat Davis, OWASA Sean Senascall, Tacoma Water

Bill Davis, CDMSmith Lorna Stickel, Portland Water Bureau Ron Harris, Newport News Waterworks David Yates, NOAA-NCAR

Rick Palmer, University of Massachusetts Paul Peterson, Arcadis

Tom Rockaway, University of Louisville Thurlough Smyth, New York City DEP Roland Steiner, WSSC

Jennifer Warner, Water Research Foundation Doug Yoder, Miami Dade Water and Sewer

Project Team

Alan Roberson and Craig Aubuchon, American Water Works Association Abby Arnold, Elana Kimbrell, and Dani Ravich, Kearns & West

Emmanuel Donkor, Refik Soyer, and Tom Mazzuchi, GWU Erik Haagenson, Balaji Rajagopalan, and Scott Summers, CU Jack Kiefer, Hazen and Sawyer

(5)

1

I. Executive Summary

Water demand forecasts are critical tools for water system managers and planners. Water system managers (and their planning and engineering staff) have to contend with many uncertainties in planning, designing, and operating a water system to meet

customers’ demands. Water demand and the resultant water sales generate the revenues that are the economic engine for any water system, whether large or small, urban or rural. Since accurate water demand forecasting is inextricably linked to a water system’s finances and the system’s long-term sustainability, the financial implications of water demand forecasting to a system are significant.

Accurate water demand forecasting depends on a variety of factors. Weather conditions, economic/business cycles, and new connections or the loss of a large industrial/commercial customer can affect short-term demands that impact day-to-day system operations. Over longer terms, other factors can influence demand, including trends in population, housing, density of land use, employment, mix of industries, water efficiency and conservation programs, and climate change and variability.

The focus of this project was to develop a better understanding of the potential risk posed by climate change and variability to demand forecasting for a water system; and then, to develop recommendations to help reduce climate-induced risk arising from inaccurate forecasts. What has been realized, however, is that long-term weather trends, caused by climate change and/or weather cycles, are only part of the picture, and water system planners, engineers, management, and water boards must invest adequate time and resources to understand all the factors that influence water demand forecasts, and the interplay between them. Moreover, the

interrelationship between the different factors is system-specific; therefore, stakeholders of each water system need to develop a better understanding of the appropriate factors for their system.

Although there is no simple answer and no clear path to reducing climate-induced risk in developing demand forecasts for water systems, this project yielded several recommendations

(6)

2 that water systems can implement now in order to develop a better understanding of climate-induced risk in water demand forecasting. Many of these recommendations involve improved collection and analysis of typical water system demand data. Some of the recommendations are time and resource intensive while other are less so. The recommendations fall into six general categories that need further investigation by water system managers to determine how to appropriately implement them in their systems, noting that many system-specific factors would impact implementation:

1. Collect additional weather and demand data.

2. Analyze the data and translate it into actionable information. 3. Evaluate potential changes in demand.

4. Evaluate potential changes in demographics in the service area. 5. Understand and incorporate uncertainty into forecasting. 6. Plan for drought so that the system can cope with it.

A water system manager faces many competing priorities in operating and managing a water system, including managing the system finances and optimizing future capital investments. The above recommendations offer a starting point for the water system manager when considering the investment of time and resources necessary to improve and optimize a system’s long-term water demand forecast.

Many utilities today still develop demand forecasts using the simple product of estimated per-capita demand and a projection of population—known as the ―per-per-capita‖ or ―gpcd‖ method. However, increasingly complex constraints in source water availability, variability in water sales and revenues, and concerns about climate change and other emerging uncertainties, have led to more emphasis on evaluating, understanding, and modeling the factors that influence water use over both short-term and long-term intervals. In particular, traditional per-capita approaches to forecasting water demand neglect—and are incapable of—measuring the effects of principal factors that can produce variability in water use, such as weather and climate, the price of water, land use, and several socioeconomic variables other than population. Additionally, the observed reductions in per-capita use, for example, due to increases in water efficiency, the effects of pricing, and recessionary economic pressures have been largely unanticipated by some systems.

(7)

3 While few could predict the impacts of the latest recession, reliance on simple forecasting

methods is partly to blame for some systems not being adequately prepared for decreases in water sales and revenues.

Long-term water demand forecasts have traditionally assumed long-term normal weather (or stationary mean) patterns for future forecast scenarios. However, if ―stationarity is dead,‖ (i.e., past weather patterns may not be the same in the future), then climate change may be particularly problematic for long-term water demand forecasts (Milly 2008). New and ―drifting‖ climate regimes may lead to changes in outdoor

watering patterns that would ultimately impact water demand, as well as other structural shifts in water use. Along the way, greater variability in precipitation and temperature in the future may also increase demand uncertainty over short-term planning horizons.

Improving forecasts is critical to optimizing future investments, as most water systems are faced with a myriad of investment decisions, ranging from finding new water sources to supply a growing population and economic growth; to additional treatment to comply with new

regulations; to rehabilitating and/or replacing distribution system pipes that have reached the end of their useful life. Water system managers need to make the investment in developing a better understanding of the potential risk posed by climate change and variability in order to improve demand forecasting for their systems so that future capital investments will be optimized.

“If ‘stationarity is dead,’ then climate change may be particularly

problematic for long-term water demand forecasts”

(8)

4

II. Introduction and Overview

The basic mission of a water utility is to meet the water demands of a community. Water demand and water sales are the primary source of a utility’s revenue. As such, water demand and the resultant revenues serve as the economic engine for a water system. Expectations, or forecasts, of water demand therefore play a crucial role in a system’s financing, as well as its short- and long-term operations. But a water utility’s role has a much greater impact on the community it serves in that the benefits of water service are fundamental for the community’s well-being and its long-term viability. Those benefits include protection of public health and safety, quality of life, and economic viability. In short, water demand forecasting also affects a community’s long-term viability, so it must be done accurately.

Every year, a water system planner predicts how much water its customers will buy (water sales) and subsequently predicts the resultant revenues as part of the annual budgeting process. The financial implications of this prediction are felt every year and a system planner finds out throughout the year the accuracy of the predictions. Lower-than-expected water sales result in lower-than-expected revenues and vice versa. Many water systems are seeing declining water sales and revenues in the face of rising operation and maintenance costs, especially for

infrastructure repair and replacement (AWE 2012). Persistent errors in short-term forecasts and revenue instabilities can result in the need to make frequent changes in rates and can also affect bond ratings and a utility’s ability/cost to borrow.

Forecasting water demands over medium and longer terms is critical for developing and funding Capital Improvement Plans (CIPs) and for planning future investments in system infrastructure. Errors and inaccuracies in long-term forecasts can result in large costs to utilities and rate payers, in the form of investments in stranded capital assets, insufficient supply reliability, or a reduced level of service due to supply and/or system capacity limitations. In some cases, state regulatory agencies could end up making decisions on future supply potentials that could significantly impact a system’s supply portfolio. In some private water rights cases, exactly who ends up holding the ―empty bucket‖ is only likely to be resolved in court. These types of problems can take considerable time and resources to resolve.

(9)

5 Because of these implications, system managers should be thoughtful about evaluating demand and invest more time and resources for the development of additional technical capacity (both internal and external) for developing more accurate demand forecasts for both the short-term and the long-term.

Forecasting and Uncertainty

Forecasts of water demand depend on a number of factors that are assumed to influence water use. If a forecast is based on a mathematical model, this means that a predicted value of demand is some function of predicted future values of important variables, such as population or job creation.

Forecasts can be inaccurate for a number of reasons. The two most fundamental reasons involve errors in

assumptions of the forecasting model and errors in predicting the future values of variables contained in the forecasting model. For example, if future water demand is assumed to be a function solely of population, but other factors influence water use, then even a perfect prediction of future population will not result in an accurate forecast. On the other hand, if it turns out that demand really does depend only on population, then an imperfect prediction of future population will also lead to inaccuracies in forecasting water use.

In reality, both of these sources of uncertainty are likely to occur, which typically makes perfect forecast accuracy unattainable (except by chance). Making predictions about the future always involves uncertainties—the key is identifying and understanding the sources of those

uncertainties. This amounts to learning more about why water demands vary, making

improvements in how forecasting models are designed, and identifying the best ways to portray forecasts that are inherently uncertain.

“Making predictions about the future always involves uncertainties—the key is identifying and

understanding the sources of those uncertainties.”

(10)

6

Climate Change as a Source of Forecast Uncertainty

Historically, several factors and trends have made forecasting water demands difficult: increased indoor plumbing efficiency; economic boom and bust cycles; fluctuations in prices for land and housing; new and emerging industries; evolving water-use attitudes; and normal day-to-day and month-to-month variability in weather. Looking forward, another complicating factor will likely be climate change and/or weather extremes.

One only needs to review the weather statistics for 2012 to see examples of climate change and extreme weather patterns. The year 2012 was the warmest of any year in the 1895-2012 period of record for the United States (NCDC 2012). Every state had above-average annual temperatures and 19 states had record warm annual averages. Much of the United States was drier than average for 2012. The area of drought in the United States during 2012 roughly equaled the drought of the 1950s. The drought peaked in July, when according to the Palmer Drought Severity Index (PDSI), 61.8 percent of the United States was in moderate drought.

The 2013 Draft Climate Assessment Report from the National Climate Assessment and Development Advisory Committee (NCADAC) provides some insight into future weather conditions (NCADAC 2013a). In this report, the NCADAC concludes that U.S. temperatures will continue to rise, with an increase of 2o – 4oF predicted for most areas. The report also concludes that the chances of record-breaking high temperature extremes will continue to increase as the climate continues to change.

For water resources and precipitation, the NCADAC concluded that precipitation and runoff increases have been observed in the Midwest and the Northeast and are predicted to continue or develop in the northern states (NCADAC 2013b). Parallel decreases have been observed and are projected to continue in the southern states. Droughts are predicted to intensify in most of the United States, with long-term reductions in water resources in the Southwest, Southeast, and Hawaii in response to both rising temperatures and changes in precipitation.

(11)

7 Changes in both average and extreme temperatures will impact demand forecasting. Long-term water demand forecasts have traditionally assumed long-term normal weather (or stationary mean) patterns for future forecast scenarios. If, in fact, ―stationarity is dead‖ as previously mentioned, then climate change may be particularly problematic for long-term water demand forecasts. New and ―drifting‖ climate regimes may lead to long-term changes in outdoor

watering patterns that would ultimately impact water demand, as well as other structural shifts in water use. Along the way, greater variability in precipitation and temperature in the future may also increase demand uncertainty over short-term planning horizons.

However, for water utilities, it should be noted that continued data collection and long-term statistics are needed before a change can be demonstrated to be attributable to climate. For some data elements such as stream flow and temperature, long-term datasets are available, but are not available for other data elements. It should also be noted that the U.S. Geologic Survey (USGS) is under significant budget pressures (like all federal agencies) and maintaining its existing network of stream gages in the future will be challenging. More data collection, research, and statistics are needed to better understand the relationship between climate and weather at a watershed level to factor into water resource planning and water demand forecasting.

The range of possible future climate conditions and weather extremes is uncertain at this time. However, it is prudent to evaluate potential changes to future water demand and demand forecasts that may be attributable to differences in future climate.

Goals and Objectives

The goal of this report is to improve water demand forecasting by increasing the awareness of water system managers and demand forecasters to the potential implications of climate change for water demand forecasting. This report is not intended to resolve the debates surrounding climate change and all of the potential implications of climate change for water systems. Rather, the objectives of this research were to:

Conduct a literature search and review of the existing research on water demand Conduct case studies using extreme value analysis on the potential impacts of climate change to water demand at two water systems (Aurora, Colorado, and Tampa, Florida) Identify knowledge gaps and research needs related to demand forecasting

(12)

8 Develop a list of recommendations for what water system managers should be doing now to improve their own demand forecasts

Conduct outreach to water systems on the need to improve their own demand forecasts

In order to address the above objectives, this report is organized as follows:

Section III examines the primary reasons to look at water demand (or why improving demand forecasts is important) and addresses the implications of climate change on water demand.

Section IV details the project approach and research methods for conducting outreach on demand issues to water system managers and experts.

Section V presents a review of the current state of water demand forecasting, existing research, and example approaches.

Section VI provides results of the Aurora and case study, and examines risks associated with model methods and climate change.

Section VII summarizes what systems are doing now and what they can be doing better to improve their own demand models.

Section VIII provides recommendations for future research. Section IX provides a summary and conclusions for this research. Section X lists the references used in this report.

(13)

9

III. Reasons to Examine Water Demand

Background

Water system managers and planners face a myriad of competing issues (AWWA 2013), such as: Rehabilitating or replacing infrastructure

Lack of public understanding of the value of water Capital costs and availability

Water supply and scarcity

Aging workforce/talent attraction and retention Regulation and government oversight

Water security and emergency preparedness Climate risk and resiliency

Managing finances and optimizing investments are two critical priorities for all water systems. All water system managers need to balance the short-term delivery needs of their customers with the long-term planning necessary to build and maintain required infrastructure, including

adequate source waters and treatment and storage capacity to meet increasing future demands.

This section provides some reasoning and justification as to why system managers should be thoughtful and deliberate in evaluating demand and why more time and resources should be invested in order to develop more technical capacity for deriving more accurate demand forecasts. The section also provides an overview of the importance of demand for system operations and planning and how demand forecasting is used (or should be used) in the strategic planning of water systems.

The Importance of Water Demand for Operations and Planning

Water demand and resultant water sales represent a water system’s economic engine. Demand projections form the basis of several complex financial and strategic decisions. Those that link to capital investments (noting that daily and seasonal projections are needed for water system operations) are usually prepared for the short term (one to five years) and the long term (15+

(14)

10 years), but each projection has a different use and each may be performed by a separate group or division of water system staff.

For example, water system staff typically develop an

operating budget for the upcoming year or financial planning period (usually five to ten years), and then this budget is refined by management and ultimately approved by the water system’s governing body. Inherent to the development of these budgets is a projection of water demand that translates into a projection of gross revenues, and, in conjunction with utility costs, projections of net revenue requirements. Additionally, if a system has sold bonds to pay for capital improvements, short-term demand forecasts are important for predicting the bond coverage ratio for future years for the system. For many water systems, meeting a specific bond coverage ratio is an important financial and strategic goal.

Meanwhile, water demand forecasts also drive long-term investment and planning strategies, which may be derived from asset management plans, and are commonly expressed in capital investment plans, system master plans, and/or urban water management plans. These plans typically require a longer view on the adequacy and reliability of the water system. These plans also detail when the next new source of water supply and/or the next increment of treatment capacity might be needed in the future.

Role of Demand Forecasting in Management and Planning

Demand projections are one of the primary methods by which most water system managers attempt to align short-term and long-term priorities and objectives. Several demand forecasting methods are available, which vary in complexity and data requirements. Adopted techniques will typically differ from utility to utility depending on a host of factors, including the adequacy of existing supplies, the diversity of the customer base, internal technical capabilities, and

(15)

11 Traditionally, water demand forecasts have been prepared using relatively simple methods, such as taking the simple product of an estimate of the per capita demand and a projection of

population—known as the ―per-capita‖ or ―gpcd‖ method. In some areas, water systems are required to use this method for demand forecasting (SFWMD 2012). In South Florida, the average per-capita daily use is calculated for the last five years or period of record. This method of calculation is

adequate for gradual decreases in per capita demand but may not adequately account for more rapid decreases.

Additionally, increasingly complex constraints in source water availability, financial capacity, and concerns about climate change and other emerging uncertainties, have led to more emphasis on evaluating, understanding, and modeling the factors that influence water use over short-term and long-term horizons. In particular, traditional per-capita approaches to forecasting water demand neglect—and are incapable of—measuring the effects of factors that can produce variability in water use, such as weather and climate, the price of water, land use, and several other

socioeconomic variables other than population. In fact, observed reductions in per-capita use— for example, due to increases in water efficiency, the effects of pricing, and recessionary pressures—have been seen by many systems over the past 30 years (Rockaway et al. 2011). In some cases, systems have modified their rate structures to account for declining water use; however, in some cases, the decline was not anticipated by some systems. Reliance on simple forecasting methods is partly to blame. Addressing factors that influence demand in the demand forecasting process permits a means in which to evaluate and consider future demand in the context of long-term investment and planning strategies.

Water demand is inherently difficult to forecast because water is a complex, multidimensional commodity that operates in legal, economic, and hydrologic dimensions (Olmstead 2010). Water is used for a variety of purposes. Some purposes are essential for public health, like drinking, cooking, and bathing. Other uses include water for cooling, irrigation, production of goods and services, and aesthetic purposes. Demand patterns vary significantly on a daily,

“Water demand is inherently difficult to forecast because water is a complex,

multidimensional commodity.”

(16)

12 weekly, and seasonal basis. Water treatment and distribution systems must be designed to meet peak demands during the height of summer irrigation season (for most of the United States), while having adequate sources to meet average annual demands. Factors that can affect short-term demands and demand forecasts include:

Weather conditions and extremes

Restrictions on outdoor use due to water shortage Economic/business cycles (such as recessions)

New connections or the loss of a particular customer (particularly a commercial or industrial customer that is a large water user, for example, a factory shuts down)

Over longer time horizons, several factors can influence demand, including trends in: Population

Housing and housing mix (e.g., single-family detached homes versus multifamily developments)

Density of land use and lot sizes Employment and mix of industries Disposable incomes and economic output Price of water and sewer service

Water efficiency and conservation programs

Re-use of treated wastewater Climate change and variability

Climate change represents a relatively new source of uncertainty in the planning process for many water systems. To date, the understanding of the potential impacts of climate change on source waters and watersheds, public health, and infrastructure investments is evolving (Means et al. 2010, AWWA Climate Change Committee 2011). These analyses have generally

considered the impacts of climate change on water supply and show that climate change is emerging as an additional consideration in the planning and design of water infrastructure. However, from a planning perspective, water supply and water demand represent the two sides

“

Climate change represents a relatively new source of uncertainty in the planning process for many water systems.”

(17)

13 of the water budget and changes in either demand or supply influence the strategic planning for the other.

One of the difficulties in incorporating climate change into demand forecasts is a clear

understanding of time frames in forecasting and the distinction between weather variability and climate change. Put succinctly, ―Climate is what you expect, weather is what you get‖ (Miller and Yates 2006). Water managers have historically dealt with weather variability in short-term operations through the use of a safety factor or supply buffer to deal with potential drought or increased demand.

However, over the long-term, excess capacity may not be available to provide sufficient buffers, or stranded capacity may lead to financial challenges in meeting debt payments and the ability to maintain high bond ratings. It is no surprise then, that a recent study on the impacts of climate change on infrastructure planning and design found that ―most [water systems] anticipate needing to make changes to their demand forecasting modeling‖ (Means et al. 2010). Figure 1 conceptualizes a few of the climate-induced changes in demand along the two dimensions of time and water system orientation.

Figure 1. Climate-Induced Changes in Demand. Source: AWWA Short Term Long Term

Inter (within watershed) Intra (within water system)

Water Rights and Demand: Minimum In-Stream Flows Agriculture Industry Power Generation Seasonal Peak (summer irrigation) Drought Restrictions

Land Use Patterns: Residential Industrial Environmental Source Water Supply: Quality

Quantity

End Use Technologies Demographics/ Population

(18)

14 Figure 2, reprinted from the Water Services Association of Australia (WSAA), illustrates several of the direct and indirect factors that influence water demand and demand forecasting.

Figure 2. Factors Influencing Water Demand. Source: Original Figure from Water Services Association of Australia Occasional Paper No. 9 – Urban Water Demand Forecasting and

Demand Management.

Implications of Climate-Induced Changes in Demand to Strategic Planning

Climate change will likely exacerbate existing pressures and situations of supply and/or demand stress (Kiefer et al. 2013 forthcoming). For example, regions that are expected to receive less precipitation and experience warmer temperatures could see a lengthening of the irrigation season and higher summer peaking factors. Coupled with population growth, some urban water systems may experience more frequent regional conflicts involving competing demands from agriculture, power production, and in-stream uses of water. Anticipating these possible changes will be important for designing long-term adaptive strategies. Thus, demand forecasts and the informational characteristics of demand models will become even more important from a strategic perspective.

The Water Utility Climate Alliance (WUCA), a committee of ten large water systems, published a white paper in 2010 on Decision Support Planning Methods (DSPMs) for water systems and how to incorporate climate change uncertainties into long-term planning (WUCA 2010). This

(19)

15 white paper presents five distinct DSPMs, with a special emphasis on the availability and familiarity of traditional scenario planning. Scenario planning allows a water system manager to better understand the risk and exposure of potential investment decisions to different scenarios based on changes in climate, demand, or business conditions. By incorporating climate change into demand projections, and allowing for both micro and macro dynamic feedback from climate change on land use, population growth, and supply availability, water managers can more accurately develop potential scenarios that model the potential variability in demand.For example, one of the earliest demand forecasts to include both conservation and climate scenarios found that in the Washington, DC, metro area, future climate-induced demand could likely be offset by

conservation programs including regulatory policies for appliance efficiency and appropriate pricing signals (Boland 1997). By more explicitly dealing with future uncertainties, this type of scenario planning will allow water systems to more appropriately justify infrastructure

investments, implement adaptation practices, and generate political support when discussing rate and cost of service studies.

Summary

Water system managers and staff need to understand both their current demand and their future demand predictions. All water systems need to balance the short-term delivery needs of their customers with the long-term planning necessary to build, finance, and maintain the required infrastructure, ranging from source waters, treatment plants, transmission mains, distribution system pipes, storage tanks, booster pumping stations, and any other needs. Accurate water demand forecasts are critical for short-term and long-term service and financial sustainability of any water system.

“Accurate water demand forecasts are critical for short-term and long-term service and financial sustainability of any water system.”

(20)

16

IV. Project Approach and Methods

The American Water Works Association (AWWA) was the lead organization for this project. Established in 1881, AWWA is the oldest and largest nonprofit, scientific, and educational association dedicated to safe and sustainable water in the world. With more than 50,000

members worldwide and 43 sections in North America, AWWA advances public health, safety, and welfare by uniting the efforts of the entire water community.

As a member-driven association, AWWA drew on the expertise of its members to provide a diversity of perspectives on water demand forecasting and the potential implications of climate change. The sections below describe the project team and the process used to solicit and obtain input related to water demand and water demand forecasting.

Project Team

A multi-disciplinary team was assembled for this project. Kearns & West (K&W), a firm specializing in stakeholder engagement, assisted in planning the two project workshops in 2011 by conducting surveys to workshop attendees to develop the priority demand/climate issues. K&W also facilitated the two project workshops and additional follow-up webinars to continue solicitation of expert judgment on the project report outline and drafts of the project report.

George Washington University (GWU) conducted a literature search and a review of the existing research and recent studies on water demand. University of Colorado-Boulder (CU) conducted two case studies using extreme value analysis on the potential impacts of climate change to water demand at two water systems (Aurora, Colorado, and Tampa, Florida). These case studies are intended as examples to guide other utilities in conducting similar extreme value analysis to feed into their improved water demand forecasts.

Hazen and Sawyer, a multi-disciplinary engineering consultant, provided technical information on water demand forecasting, and provided additional information during the editing of this report.

(21)

17

Pre-workshop interviews

AWWA engaged a broad range of stakeholders throughout the study. The stakeholder input helped inform the study of issues to be addressed in the topic of water demand and to develop recommendations on how to address these issues. AWWA, working with K&W, conducted interviews with representatives from key stakeholder groups prior to the two workshops in 2011 and designed the workshops based on stakeholder input. Additionally, in order to inform the discussion at the two workshops, AWWA and K&W conducted an electronic survey of workshop participants. Below is a summary of the interviews, the survey results, and the workshops.

K&W interviewed eight experts, representing consultants (3), academics (4), and utilities (1). The experts were interviewed on a number of topics including their area(s) of expertise, key themes related to the current state of

modeling, suggested topics to consider during the

workshops, and recommendations for additional participants.

From the interviews, K&W found that overall, the quality of water demand forecasting models is highly dependent on funding and data availability. Lack of data in a consistent format from different sources and different data systems is a significant obstacle to developing a useful model for a utility, as well as different data formats hindering the development of national-scale

models. Many existing models are not able to adequately define important variables and/or disaggregate water use spatially or by sector (e.g., by region, family versus multifamily,

commercial versus industrial). Also, some utilities do not use mathematical or statistical models because they do not see the need, do not have staff trained to use them, and/or data is not readily available in a format that is easy to model.

In general, the interviews suggest there is a need for standardized data and possibly a central portal for data access, as well as models that account for climate change (using regional weather patterns). More modeling expertise and funding is needed, as well as the inclusion of behavioral variables in demand forecasting, and a better understanding of industrial water use.

(22)

18 The interviews indicated that increasingly, larger utilities are seeking to model socioeconomic and demographic variables as drivers in their demand forecasts. However, the majority of utilities, especially the smaller utilities, still focus on cost as the main driver underlying the characteristics of their demand models and demand forecasting approaches.

Surveys and Workshops in Two Regions

K&W assisted AWWA in hosting two workshops, one with an East Coast focus and the other with a Midwest/Western focus, to help understand respective concerns and recommendations, and how they differ from each other.

East Coast Focus in Washington, DC

The first workshop was held on March 30, 2011, in Washington, DC. The goal of the workshop was to initiate an interactive dialogue, and to give presentations on methodologies used in forecasting water demand and climate change. Prior to the workshop, K&W sent a ten-question online survey to participants, to generate thinking and help inform discussion at the workshops. There were 14 respondents to the survey sent in advance of the East Coast meeting. Participants in the survey and workshop were primarily from the Washington, DC, area, Florida,

Massachusetts, and New York. The following summarizes the results of the East Coast survey, and then identifies the objectives of the East Coast workshop and some of the main discussion points and recommendations.

Pre-Workshop Survey

Participants of the East Coast survey indicated that the main reasons people use water demand forecasts are for water conservation, revenue forecasts, long-term supply planning, regulatory planning, and infrastructure planning. The climate-related concerns raised were based on potential changes in water flow, duration, and intensity. The survey results indicated concern about extreme events, timing of floods and drought, changes in stream flow, sea-level rise, and changing consumption patterns. Forecasts in the East generally use population demographics combined with per-capita data, based on billing information. The results from the East Coast survey indicated that sophisticated models are not widely used, and historical climate data is

(23)

19 used minimally by the majority of respondents. The water demand data that people are most interested in are peak daily demand and total annual demand. The survey found that most people wanted to attend the workshops to learn new methodologies and best practices for improving their water demand forecasting capabilities, including how to factor in climate change.

Workshop

The following topics and recommendations were discussed at the March 30, 2011, workshop in Washington, DC:

1. Review existing research on water demand related to climate change Presentations were given on related topics

2. Discuss current models’ strengths and weaknesses

It was suggested that models break down data by customer type, and account for population densities

3. Identify knowledge gaps and list future research topics

Share best practices for data collection and model methodologies; standardize the data collection process; and develop data templates

Study the effects on water demand of new water efficient fixtures; utility water use (such as continuously running water through pipes to prevent freezing); reuse of water; and, green building practices

Predict demographic and behavioral responses to climate change (rising sea levels) Consider utility zoning

Include population density in models

Look to the financial and insurance industries/communities for lessons learned about diversification and risk

Partner with energy utilities

4. Develop recommendations for how water utilities can reduce the uncertainties in water demand forecasting

Better acknowledge and communicate the possibility of error and the confidence intervals to decision makers

(24)

20 Focus on how to incorporate climate change into a forecast; potentially compare

climate change models to both seasonal forecasts and longer term supply forecasts

Midwest/Western Focus in Denver, Colorado

The second workshop, held on July 12, 2011, in Denver, Colorado, was focused around the same main topics as the workshop held on the East Coast and many of the same issues and

recommendations were discussed.

Pre-Workshop Survey

The Midwest/Western survey sent in advance of the July workshop had nine respondents, who primarily reside in the intermountain west. Participants stated that their main reasons for using water demand forecasts were for long-term and short-term planning, which encapsulate many of the same purposes noted in the East Coast survey. In the West, complex models seem to be used more frequently, and historical climate data more commonly incorporated, in order to

characterize and model drought cycles and decreasing stream flows. Climate concerns included extreme events, stormwater runoff, and effects of precipitation on infrastructure. The water demand data that people are most interested in are total annual demand, followed by peak daily demand. The other results of the survey were largely the same, including the respondents’ goals for the workshop.

Workshop

Additional notes from the July 12, 2011, workshop included the following:

1. Climate change models are often too complex to be useful for decision making; there is a need to develop ―actionable science‖

2. Develop recommendations for utilities based on a ―profile type,‖ depending on their local/regional issues, their size, funding, etc.

3. Consider alternative water sources 4. Improve tools to analyze data

(25)

21 After the two workshops, a series of meetings were held via webinars to review the draft report sections on the priority research plan and the recommendations for water systems. These webinars were held in October and December of 2011, and in January, April, and May of 2012. Through an interactive format, these webinars provided a mechanism for the workshop attendees to provide additional input on the recommendations for what utilities should be doing now to improve water demand forecasting and on recommendations for future research. Additional input was also given through reviews of the draft report.

(26)

22

V. Current State of Water Demand Forecasting

Basics of Water Demand Forecasting

There are a number of ways to forecast water demand, which vary in analytical rigor and requirements for data. In the most general sense, the intent of a demand forecast is to make a prediction of future water use. However, the actual dimensions of the problem can be numerous and more complex depending on considerations related to agent or purpose specificity, temporal scale, and spatial extent (Kiefer et al. 2013). In other words, how water use is defined (e.g., total use in a service area, water use of particular user types or sectors or geographic areas, annual, monthly, or seasonal demand) will influence the choice among alternative demand modeling and forecasting methods.

At a fundamental level, a demand forecast represents a set of calculations, which defines a formula and embodies a set of assumptions. In short hand, one can generalize the set of

calculations symbolically as a function, Q=f (X), where Q is the measure of water demand to be forecasted, X represents a set of factors that are part of the calculation and thus influence the forecast, and the term f(*) defines mathematically how X relates to Q. Therefore, future or forecasted values of Q are a function and conditioned on future or forecasted values of X. Unfortunately, and in the case of most situations that involve human preferences and choices, neither the ―true‖ nature of the dependencies on X are seldom known with certainty nor is the proper definition of X. Even if one is skilled or lucky enough to have f(*) defined properly, then one must have confidence in forecasts of X to have confidence in the forecast of Q. Limitations on available data, less than perfect knowledge of underlying relationships, and inherent

uncertainty about the future make water demand forecasting both an art and a science.

Demand Forecasting Methodologies

Based on reviews of contemporary demand forecasting approaches found in Kiefer (2006) and Billings and Jones (2008), one may classify demand forecasting into several different categories. The Aggregate Per Capita Approach is a traditional approach to water demand forecasting that relies exclusively on population projections. The aggregate per capita approach assumes a fixed

(27)

23

Q = f(X)

value of water use per person (per-capita consumption) and multiplies this value by population to calculate a forecast. (Using the formula previously discussed, the definition of X includes

population, an estimate of water use per person, and f(*) represents a simple multiplication of these terms.)

Other Fixed Unit Use Coefficient methods define other fixed water use factors and drivers of demand (other than population) to prepare a forecast. Examples of these methods include the use

of water use per acre coefficients and

projections of future developed acres, water

use per residential housing unit coefficient and

projections of future housing units, and water

use per employee coefficients and

projections of future employment. Many of these types of forecasts rely on disaggregation of water use into user sectors and seasons, which may improve the informational qualities of a forecast relative to the per-capita method. However, similar to the per capita method condition, the demand forecast relies solely on counts of users or related proxies.

Time-Series and Trend-Based Models predict future water demand based on assignment of trend parameters or statistical (autoregressive) relationships that link past values and systematic repeating cycles of demand to future values of demand. Using the conceptual example above, past values of Q are used to predict future values of Q, and the function f(*) defines how past values relate to current and future values. Time-series models tend to be used to predict demands over relatively short timeframes when longer-term influences may not be as significant.

Regression and Econometric Models are statistical models that explicitly estimate the parameters of a function that relates changes in defined explanatory variables (X) to changes in water use. This class of models uses cause-effect relationships among water use and specified factors that affect water use to forecast demand. Econometric models can be considered a class of regression models that specify variables, such as price and income, which according to economic theory, would be expected to influence consumption.

End-Use Models account for and forecast water used by specific water-using fixtures, appliances, or for specific purposes. In most cases, end-use models represent an accounting

(28)

24 structure that is dependent on assumptions for water-using technologies, market saturation of various water-using technologies, and behavioral factors, such as frequency of use. Some end-use models are developed using regression analysis that specifies factors correlated with end-end-use consumption (e.g., see Mayer et al. 1999). These types of models are well-suited for examining the effects of increasing water efficiency through time, which result from plumbing standards and codes and water utility conservation programs.

There can be considerable overlap among some of the forecasting approaches classified above, particularly with regard to disaggregation of water use sectors, as well as hybrid models that blend the features of different techniques. For example, unit use coefficients may be scaled according to information and parameters obtained from the literature or by means of separate regression models (sometimes called variable forecast factor approaches). Time series models may also be blended with regression models to create forecasts based on past values of

consumption and exogenous factors. Furthermore, outputs from end-use models are sometimes used to adjust the results of forecasts derived from other methods in order to account for predictions of future water efficiency. Finally, traditionally less conventional methods, such as artificial neural networks, are being used more often to model and characterize (or learn) patterns of water consumption, which may hold some promise for demand forecasting.

How Factors that Affect Demand Are Addressed in Models

Some of the key factors that affect water demand were previously described in Section III. By construction, different models will have different capabilities for addressing these factors or will address them differently. For example, by relying only on population, the per-capita approach cannot directly address factors other than population that influence water demand, and cannot recognize differences in water use patterns across water use sectors. Sector-based fixed

coefficient methods may provide additional information on the structure of underlying demands, but will generally also lack the ability to test and specify the effects of other factors, especially economic and climatic factors.

Regression model and econometric approaches are able to incorporate multiple variables to explain and predict water demand. However, the degree to which the multiple factors that

(29)

25 limited to the availability of historical data to estimate numerical relationships and the existence of projection data to effectively use these factors for the purposes of forecasting. Nevertheless, predictive model-based approaches to water demand forecasting would seem to be the most ideal for evaluating the potential effects of climate change. Notwithstanding data constraints, they are capable of directly incorporating principal indicators of weather and climate, which can be used to assess alternative scenarios. Disaggregation of data into water use sectors and time periods further augments the capability to analyze climate change by providing an opportunity to isolate impacts on the underlying components of water use.

How Uncertainty Is Addressed in Models

Water demand forecasting involves inherent uncertainties. As suggested in earlier sections, there is always incomplete knowledge and understanding of the determinants of water use and how they are best related in a mathematical sense to water use. In addition, future values of important factors are not known with certainty and/or can be highly variable, which makes it virtually impossible to achieve 100 percent forecast accuracy. In fact, even if one were to know the future values of key factors with certainty, a demand forecast is likely to be wrong because of practical and technical shortcomings related to the model being used. For example, if one were to predict future population with 100 percent accuracy, the per-capita forecasting method may still produce an inaccurate forecast because water consumption depends on more than just population. This example rightly implies that the options available to address uncertainty are also affected by choices about the design of the forecasting model.

In practice, and depending on the characteristics of the forecasting model, forecast uncertainty is addressed through the use of scenarios, the application of statistical routines for estimating forecast error, or both. Scenario analysis is very common, such as using high, medium, and low population growth to create an envelope of future demands. Other types of scenarios such as hot-dry, cool-wet weather scenarios are also often used, but require a model or other mechanism to translate weather into predictions of water use. Oftentimes, extreme scenarios are combined in an attempt to account for most future demand possibilities.

(30)

26 Standard formulae exist to

calculate random and sampling error associated with ordinary least squares regression procedures. However,

computational difficulties have tended to limit applied uncertainty analysis to evaluation of

conditional forecast error, which assumes the model is accurate and accounts only for uncertainty about the future values of model variables. Monte Carlo simulation methods are sometimes used to simulate potential values of independent variables, given some underlying knowledge or

assumptions regarding the type and shape of their respective distributions, which results in a range of predicted demands.

Current State of Water Demand Forecasting Models

This section summarizes the results of a literature review on water demand forecasting,

conducted by researchers at The George Washington University (GWU) as part of this project. The objective of the GWU research was to provide a guide to the literature on improving the practice of demand forecasting for effective decision making.

GWU conducted a search of the water demand forecasting literature published from 2000 to 2010 and developed a bibliography of 79 papers from a cross-section of peer-reviewed journals. These papers were then categorized into the appropriate type of model (qualitative extrapolative methods versus nonparametric). The analysis of these models focused on three questions:

How practical are the models?

Figure 3.Forecastuncertainty can be analyzed using a statistical demand model and assumptions about the distributions of variables that affect water demand. Source: construct developed by Jack C. Kiefer.

(31)

27 Are the forecasts reliable?

What is the best approach?

These papers were then synthesized in order to identify what the main focus of research has been and to make proposals on how the practice of water demand forecasting can be improved. The synthesis found that while a wide variety of methods and models have been used and have attracted attention, applications of these models differ, depending on the forecast variable, its periodicity, and the forecast horizon (Donkor et al. 2012).

The analysis found that a shift is ongoing from pure conventional methods to a focus on three approaches:

1. Scenario-based and Decision Support System (DSS) models: approaches that accommodate some amount of uncertainty in demand forecasting

2. Comparative assessment of performance between neural nets and conventional methods 3. Recognition of the need to improve forecast accuracy by using hybrid models

The results of the literature search and analysis indicated that it is difficult to answer the question ―Which model is best for water demand forecasting?‖ without specifying the periodicity of the demand variable. The research found that neural networks and hybrid models are more

appropriate for short-term forecasts; but, for extended ones, where incorporating future scenarios of a variable might be important, scenario-based and DSS models are more suitable. However, the use of regression in modeling monthly demand follows the generally held view that short-to-medium-term demand is typically influenced by weather variables while long-term forecasts are more determined by socioeconomic factors.

Overall, improving forecast accuracy, accounting for uncertainty in long-term forecasts, and maintaining system reliability now and in the future seem to have provided the impetus for the current research in urban water demand forecasting.

(32)

28

Summary

There are differences in how water system planners model and forecast demand for water. The forecasting techniques vary in their sophistication and methodology to account for determinants of water use. The choice of a particular forecasting methodology is affected by the data used to model relationships among demand determinants and sector water use. The availability and quality of data to support the development of models typically serves as a practical constraint on the options that are applicable for forecasting. Furthermore, the specific goals and objectives of any particular water demand forecasting effort may not immediately require one to enhance the prediction and informational capabilities associated with more complex methods. However, the array of uncertainties facing the water utility industry seems to require an emphasis on better forecasting and more robust modeling capabilities.

(33)

29

VI. Risks Associated with Models and Methods

Any forecast has some chance of being incorrect due to the fact that a forecast is an attempt to predict the future. Water system planners and managers need to understand and mitigate the risks of being wrong in either direction when predicting future demands. A demand forecast that turns out to be high can result in stranded capacity and the

water system paying for debt for facilities that are not producing the predicted revenues. A demand forecast that turns out to be low can result in lower than

desirable levels of service, i.e., restrictions that might be placed on water use might be unpopular with the

system’s customers.

The objective in water demand forecasting is to minimize the risk of being incorrect and to provide for adaptive management so that the water system can accommodate the range of potential outcomes and their probabilities of occurrence. Water system planners and managers now need to incorporate the potential changes in demand from climate change, and incorporate the uncertainties in future weather predictions with all of the other uncertainties previously discussed, including population and employment predictions and changes in per-capita demand.

Limitations of Existing Models

As previously discussed, models are highly dependent on the quality of the data used to build and validate the model. Improving modeling at a water system is an investment decision as it takes additional resources to go beyond what is already being done. In other words, water system managers need to evaluate whether the limitations in existing demand models are significant enough to warrant the additional investment in the collection and analysis of existing data, and in the development of improved models.

In most cases, the additional investment is justified. As previously discussed, for many water systems, demand forecasting is simply multiplying the gallons per capita per day (gpcd) by the projected population growth and job growth. However, the traditional per-capita approaches to forecasting water demand neglect and are incapable of measuring the effects of principal factors that can produce variability in water use, such as weather and climate, the price of water, land

“Each water system has a unique set of data; there is no single model that can fit all systems.”

(34)

30 use, and several other socioeconomic variables other than population. Past observed reductions in-per capita use—for example, due to increases in water efficiency, the effects of pricing, and recessionary pressures—have been largely unanticipated by many systems. Therefore, in many cases, the additional investment in the collection and analysis of existing data, and in the

development of improved models is warranted to overcome the limitations of existing models.

One Potential Approach to Identify Risks—Extreme Value Analysis

Water system managers and planners are particularly interested in the high-impact, low-probability water demand events that drive infrastructure investment decisions and the need to fund such investments. Accurate predictions of peak-day and peak-hour demands are necessary for planning capital improvements such as alternate sources of supply, treatment plant capacity, transmission mains, storage tanks, and booster pumping stations.

These events, by definition, are ―extreme events,‖ and one approach that this research found to model these events is the use of extreme value analysis (EVA). Climate change and more extreme weather events will likely change the above peak demands, and will need to be appropriately considered by water systems in future planning, design, and operations of their systems. EVA has been used in a wide variety of disciplines including the financial industry, the global reinsurance industry, civil engineering, ecology, water quality, and especially climatology and hydrology. EVA has been used in hydrology to estimate and forecast flood frequency, model financial loss related to flooding events, and to model extreme hydrological events in various watershed sizes, but has seen limited use in the water sector, particularly for demand forecasting. Research conducted by the University of Colorado-Boulder (CU) as part of this project using EVA show one potential approach to identify and evaluate risks (Haagenson et al. 2013). The objective of the CU research was to apply EVA techniques to water demand data at two case study utilities (Aurora, Colorado, and Tampa, Florida) and show the potential impacts from climate change on the water demand forecasts for these two case studies.

Focusing on Aurora, Colorado, the CU research used an EVA approach to predict the changes in water demand due to potential climate change scenarios. Daily production data from 1990-2010 showed the critical season of high demand in June-August. Daily weather data were used to develop weather attributes (hot/dry, wet/cold spells along with average weather) for

(35)

June-31 August. A simple bootstrapping method was used to forecast weather trends, and then EVA was used to generate projections of water demand extremes. This research found that under climate change scenarios, exceedances increase over time for the warm/wet and the warm/dry cases, relative to natural variability.

Summary

A forecast for anything has some chance of being incorrect due to the fact that any forecast is an attempt to predict the future. Realistic forecasts of water demand extremes should be valuable to water system managers (and their planning staff) during costly infrastructure decisions, as the cost of being wrong could be significant. Different methods and models can be used for forecasting, ranging from a simple scenario of 10 percent additional peak-hour and peak-day demands, to a slightly more complicated scenario of 10 percent additional demand coupled with a 10 percent decrease in water supply, to more computational-intensive approaches such as EVA.

(36)

32

VII. What Utilities Should Be Doing Now

Water system managers are increasingly confronted by a variety of challenges. These challenges include an increase in drinking water regulations from the Environmental Protection Agency (EPA), an aging/transitioning workforce, and increased needs for investment in the aging distribution system in the face of opposition to raising rates, especially in light of current

political and economic conditions (AWWA 2013). Business factors are a significant concern for water system managers, but climate change introduces a new set of challenges for water system managers and their planning staff for both long-term planning and for future operations and maintenance of the water system.

A universal cookie-cutter approach for incorporating potential impacts from climate change into water demand forecasting cannot be easily developed. Situation-specific approaches need to be developed for demand forecasting that take into account local considerations in terms of:

Availability of water

Characteristics and patterns of water use and related data

Characteristics of demand (sensitive to climate change) and influential explanatory factors such as the temporal and spatial characteristics of temperature, precipitation, and socioeconomic factors

Availability of internal and external modeling expertise

Understanding issues faced by management, the public, and the political leadership Real or perceived importance of climate change relative to other planning challenges (e.g., lack of new water sources) and objectives (e.g., reliability of existing supplies).

The last bullet warrants some additional discussion. At any given time, a water system is presented with a number of risks that vary in terms of immediacy, severity of potential consequences, and likelihood of occurrence. These risks can include the possibility of supply loss (due to contamination, regulation, supply seasonality/drought, turbidity, reservoir operations constraints, etc.), system or component failure, supply contract performance, stranded resources,

“A universal cookie-cutter approach cannot be easily developed.”

(37)

33 political environments, personnel/labor relations, demand management requirements, and rate affordability. Managers must be cognizant of the costs associated with eliminating or mitigating any particular risk, and must justify risk-management efforts accordingly. Realistically,

achieving zero risk is not possible and eliminating certain risks may be cost prohibitive. Decisions about how to manage the potential demand-side risks related to climate change will

need to be made within the context of the larger

portfolio of risks for each water system, and these risks typically will be different for each system.

Water systems may face several potential impacts from climate change on operations and maintenance for both existing and future facilities. For example, sea-level rise may result in a system having to relocate facilities and/or modify operations, and water demand may also change due to customer relocation and changes in regional growth patterns. Increased weather variability resulting in increased incidence of flooding, increasing numbers of other significant weather events such as hurricanes, ice storms, etc., may also have a more pronounced direct impact on the water system (i.e., flooding and/or other physical damage to the system). The relative significance of these impacts will vary from utility to utility.

Water systems need to understand that if ―stationarity is dead,‖ the past may not be the best predictor of the future, but it is important to understand past dynamics and use that information to help inform future decision-making (Milly et al. 2008). While all models are built upon past data, improving the measurement of the impacts and causal relationships between the factors that are known and have been experienced and a system’s demand is critical. A system’s average demand could change in the future due to increased market penetration of low-flow plumbing fixtures. A system’s peak demand could also change due to changes in the weather.

In the future, climate and weather conditions are likely going to change for water systems. Furthermore, future climate and weather variations may vary geographically. Some examples of potential changes that may affect operations and planning include:

“…relying solely on the past to predict future water demands could be problematic, especially without a more in-depth understanding of what factors have influenced or determined past patterns.”

(38)

34 Springtime beginning earlier and ending later, which extends the watering season

Changes in total precipitation, or the annual/seasonal distribution of precipitation (e.g., fewer but more intense storms)

Warmer temperatures and longer periods of hot and dry spells

The availability of adequate water resources (and/or the lack of new water sources) generally places water systems into three categories when considering actions to address the potential for altered conditions and associated risks. The level of effort and expense (time and resources) allocated toward increasing the understanding of water use patterns and improved demand forecasting may be characterized as:

1. Wait and See—refers to systems having ample long-term water supplies and adequate treatment and transmission capacity.

2. Start Thinking About It—refers to systems that are looking for a ―no-regret‖ strategy that can be adopted now with minimal cost, while learning more about potential impacts of climate change to their systems. Most systems in this category are also looking for flexible adaptive management strategies that can be adjusted as more data is available and translated into actionable information.

3. Should Be Thinking About It—refers to systems already resource constrained (i.e., resource shortages already exist) or nearing safe yields and where the possibility of constraints or demand pressures is likely to worsen.

More research is needed to understand specific potential impacts to water systems so that water systems can establish their risk tolerance and adaptive management positions. The research needs are described in greater detail in the next section of this report. In the meantime, system managers can take measures now to help reduce uncertainty in forecasting water demand. These recommendations were developed by a broad range of stakeholders, including water system managers, consulting engineers, and academics during the two workshops discussed in Section IV and the subsequent webinars.

Decreasing Climate-Induced Water Supply Risk Through Improved Municipal Water Demand Forecasting

Decreasing Climate-Induced

Water Supply Risk Through Improved

Table of Contents

Acknowledgements

I. Executive Summary

II. Introduction and Overview

Forecasting and Uncertainty

Climate Change as a Source of Forecast Uncertainty

Goals and Objectives

III. Reasons to Examine Water Demand

Background

The Importance of Water Demand for Operations and Planning

Role of Demand Forecasting in Management and Planning

“

Implications of Climate-Induced Changes in Demand to Strategic Planning

Summary

IV. Project Approach and Methods

Project Team

Pre-workshop interviews

Surveys and Workshops in Two Regions

East Coast Focus in Washington, DC

Midwest/Western Focus in Denver, Colorado

V. Current State of Water Demand Forecasting

Basics of Water Demand Forecasting

Demand Forecasting Methodologies

Q = f(X)

How Factors that Affect Demand Are Addressed in Models

How Uncertainty Is Addressed in Models

Current State of Water Demand Forecasting Models

Summary

VI. Risks Associated with Models and Methods

Limitations of Existing Models

One Potential Approach to Identify Risks—Extreme Value Analysis

Summary

VII. What Utilities Should Be Doing Now