EM&V: Energy Efficiency Program Evaluation,
Measurement, & Verification
Jonathon Jackson [email protected]
Agenda
Why are Energy Efficiency Programs Evaluated?
What does an Evaluation Consist of ?
How does the Evaluation affect the Energy Efficiency
Programs?
Ameren Illinois Energy Efficiency Programs
Residential Programs Lighting REEP Appliance Recycling Multifamily Home Energy Performance HVAC
Moderate Income Home Energy Reports Energy Star New Homes
Business Programs Standard Custom
Public Utilities Act Section 8-103 & 8-104
• (a) It is the policy of the State that electric utilities are required to use
cost-effective energy efficiency and demand-response measures to reduce delivery load. Requiring investment in cost-effective energy efficiency and
demand-response measures will reduce direct and indirect costs to consumers by decreasing environmental impacts and by avoiding or delaying the need for new generation, transmission, and distribution infrastructure.
http://www.ilga.gov/legislation/ilcs/fulltext.asp?DocName=022000050K8-103
• (a) It is the policy of the State that natural gas utilities and the Department of Commerce and Economic Opportunity are required to use cost-effective energy efficiency to reduce direct and indirect costs to consumers.
Utility Goals
(b) Electric utilities shall implement cost-effective energy efficiency measures to meet the following incremental annual energy savings goals:
(1) 0.2% of energy delivered in the year commencing June 1, 2008; (2) 0.4% of energy delivered in the year commencing June 1, 2009; (3) 0.6% of energy delivered in the year commencing June 1, 2010; (4) 0.8% of energy delivered in the year commencing June 1, 2011; (5) 1% of energy delivered in the year commencing June 1, 2012; (6) 1.4% of energy delivered in the year commencing June 1, 2013; (7) 1.8% of energy delivered in the year commencing June 1, 2014; and
(8) 2% of energy delivered in the year commencing June 1, 2015 and each year thereafter.
Net Program Impacts
(f)....
(7) Provide for an annual independent evaluation of
the performance of the cost-effectiveness of the utility's portfolio of measures and the Department's portfolio of measures, as well as a full review of the 3-year results of the
broader net program impacts
and, to the extentpractical, for adjustment of the measures on a going-forward basis as a result of the evaluations. The resources dedicated to evaluation shall not exceed 3% of portfolio resources in any given year.
Ameren Illinois Plan 2 Goals
Energy Efficiency
Annual MWH Savings Annual MW Savings Annual Therm Savings
PY4 PY5 PY6 PY4 PY5 PY6 PY4 PY5 PY6
RES-Lighting 82,485 61,974 42,418 2.5 1.9 1.3 0 0 0
RES-Efficient Products 11,079 11,999 13,110 2.3 2.4 2.7 324,590 463,622 552,133
RES-HVAC 13,448 14,187 15,109 6.4 6.8 7.2 896,800 1,147,316 1,480,704
RES-Appliance Recycling 19,889 20,070 16,036 2.9 2.9 2.3 0 0 0
RES- Home Energy Performance 2,593 2,665 2,728 0.7 0.7 0.7 100,890 103,916 107,034
RES-New Construction 273 304 329 0.1 0.1 0.1 12,831 14,268 15,449
RES-Multi-family 4,874 5,217 5,285 0.9 1.0 1.0 247,116 290,831 313,078
RES-Home Energy Reports 21,705 21,705 21,705 4.9 4.9 4.9 664,517 664,517 664,517
RES-Moderate Income 1,732 1,774 1,800 0.5 0.5 0.5 64,850 66,795 68,799
RES-Voltage Optimization * 0 0 0 4.5 4.5 4.5 0 0 0
RESIDENTIAL PORTFOLIO TOTAL 158,078 139,895 118,521 25.5 25.6 25.1 2,311,593 2,751,267 3,201,714
BUS-Standard 47,815 40,648 37,334 20.2 17.2 15.8 1,145,345 1,306,813 1,429,883
BUS-Custom 55,620 54,490 50,648 16.3 15.9 14.8 189,043 210,919 223,281
BUS-RCx 3,309 3,196 3,019 0.8 0.8 0.7 5,654 5,002 4,651
BUS-New Construction 8,194 7,123 6,454 2.9 2.5 2.2 51,483 50,035 47,131
Evaluation is the Assessment of Energy Efficiency Program
Performance
• Ameren Illinois contracts with an independent evaluator to perform the evaluation of our programs annually.
• Evaluations are comprised of two main components: Process & Impact • Ameren Illinois is required to have a minimum of one Process and Impact
Process
Process evaluation provides information on how the program is working in the market and is used to inform adjustments in program design and
implementation strategy. Examples of questions addressed in the Process Evaluation.
i. What are the participant characteristics? How many HVAC units were installed? At what SEER levels? What percent was early-replacement vs. replace on burnout? Have the ratios changed since last year?
ii. Has the program as implemented changed compared to previous year? If so, how, why, and was this an advantageous change?
iii. How satisfied are customers with the program? Are they likely to
recommend the program to friends or family? Are customers aware of other AIC programs?
Impact
Impact Evaluations measure quantitative results(usually energy savings) and are used to assess program performance against goals. Examples of goals set forth in the Impact Evaluation.
i. Conduct onsite verification with a subsample of the impact sample. 1. Verify measures are installed and operating. Possibly meter
equipment to verify usage and savings.
Evaluation Reports
Ameren Illinois Evaluations are public documents. Two ways you can access the documents.
1. They are publicly available on the Illinois Commerce Commissions e-docket. You will need the savings docket number to access.
2. All the utilities and interested parties participate in a group called the
Stakeholders Advisory Group. They are available for download on their website at http://www.ilsag.info/evaluation-documents.html
Program Attribution
•Evaluation Consultants
Program attribution refers to energy impacts that can confidently be attributed to
program efforts
A net-to-gross factor that reflects program attribution
is applied to gross program energy savings to
Net To Gross (NTG or NTGR)
Free Riders
Free rider is an individual that would have installed an energy efficiency measure without any program incentive but receives a financial incentive or rebate anyway.
What is Spillover?
Spillover is when an individual chooses to implement additional energy efficiency measures due to the influence of the program without an incentive or rebate.
Estimating NTG
• Not always as easy as for this respondent • Particularly difficult for Upstream Programs
Goal Achievement
When verifying the utilities have achieved the state mandated goals, the commerce commission looks at our NET results from our annual evaluations for each program.
Measure Ex Ante Gross Savings (MWh) Realized Gross Savings (MWh) Realization Rate PY3 NTG Ratio Net Energy Savings (MWh) Net Demand Savings (kW) Recycled Refrigerator 8,676 7,847 90% 0.79 6,199 772 Recycled Freezer 2,471 2,328 94% 0.82 1,909 238 Recycled Room AC 13 13 100% 1.0 13 5 Total - PY3 11,160 10,188 91% 0.80 8,121 1,015
Evaluation is used to determine measure level savings
The Illinois Utilities have collaborated to create the Illinois Statewide Technical Reference Manual (TRM)
It’s a 675 page document that standardizes the calculations of savings for energy efficiency measures
The TRM goes through an annual update procedure
It can be located on the Illinois Stakeholders Advisory Group Website
TRM Algorithm for an Energy Star Compact Flourescent Lamp
E
LECTRICE
NERGYS
AVINGS∆kWh = ((WattsBase - WattsEE) / 1000) * ISR * Hours * WHFe
For example, for a 14W CFL (60W standard incandescent and 43W EISA qualified incandescent/halogen) purchased in 2014.
ΔkWH1st year installs = ((43 - 14) / 1000) * 0.722 * 1000 * 1.06 = 22.2 kWh (GROSS)
ISR
Program Weighted Average 1st year In Service Rate (ISR) 2nd year Installations 3rd year Installations Final Lifetime In Service RateRetail (Time of Sale) or
Efficiency Kits 72.2% 13.9% 11.9% 98.0%
Direct Install 96.9%
1
styear in service rate is based upon review of PY3-5 evaluations from ComEd and
PY5 for Ameren (see ‘IL RES Lighting ISR.xls’ for more information. The average
first year ISR for each utility was calculated weighted by the number of bulbs in
each year’s survey. This was then weighted by annual sales to give a statewide
assumption.
Evaluation Drives the Program Design
Examples • Applications • Measures Offered • Incentives • Marketing ChannelsEvaluation Drives the Program Design
• The results of the annual evaluations are used in the annual Statewide TRM update procedure
• These variables drive the savings that can be claimed from each measure
• The energy efficiency programs are on a fixed budget and must achieve program targets while balancing available budgets.
Participation in Evaluation
Customers and Program Allies will be called upon to participate in evaluation through a variety of methods.
I. Phone Surveys
II. Onsite Verification
III. Metering Studies IV. Customer Intercepts
Evaluation Activity
Typically the outside evaluators perform their activities after the program year is over.
That is not always the case as we do perform some surveying activities during the program year and certain metering studies require being in the field during specific times of the year. Ex Heating Season or Cooling Season for metering HVAC equip.
Certain residential programs tend to have a consistent participation and allow for early surveying. C&I customers have varied project sizes which makes it more difficult to do early surveying since sampling is based on savings
Customer Surveys
• Customers will be contacted for both Process and Impact Surveys. The majority of the time evaluators will try to combine both.
• We try to be cognizant of our participants time. Certain surveying activities will provide an incentive for participating such as a gift card.
• The surveys will be designed to answer the questions that are posed in the workplan. See examples on Process and Impact slides.
Example Questions
• Are you aware of the fact that federal standards for lighting equipment recently changed so that there are now restrictions on the production of T12 lamps for sale
in the U.S? (Process)
• What are lighting contractors and suppliers telling you about these changes to
federal standards? (Process)
• When did you first learn about Ameren Illinois ActOnEnergyProgram? Was it BEFORE or AFTER you first began to THINK about implementing this measure?
(Impact)
• Were there any other factors we haven't discussed that were influential in your
