On-Site Survey Implementation

The on-site survey implementation process incorporates many steps. The on-site survey task manager will develop protocols and training materials and the field personnel will be trained in the implementation of the surveys. Following the training process, full implementation will gradually begin. During the phase in of the implementation, the data collected by the field personnel will be reviewed immediately following data collection; if issues arise they will be corrected prior to the personnel receiving additional sites.

Develop On-Site Protocols

This sub-task will entail the development of protocols for the collection of the on-site survey data. The protocols will help to guarantee consistency, accuracy, and efficiency in the collection of the on-site data. The 2006 CEC CEUS and the 2006-2008 Small Commercial Evaluation Protocols provide the basis for the new CMST and CSS protocols. The protocols will be developed by the research team and reviewed by the interested parties.

21 The 84% of energy consumption does not include the energy consumption associated with TVs. Televisions would have been included as part of miscellaneous in the former CEUS and this share of energy was not included in the 84%. Therefore, it is possible that the share of energy consumption included in the CSS exceeds 84%.

Develop On-Site Training Information

The research team has extensive experience developing training manuals to provide guidance to the surveyors throughout the fieldwork. The existing training manuals for the CEC CEUS and the 2006-2008 Small Commercial Evaluation will be reviewed, integrated, and customized to address the unique features of the CSS and CMST surveys. At a minimum, the manual will be updated to reflect the new CSS and CMST surveys. The manuals will also be adjusted to reflect the new material that the research team has found or developed that help explain the equipment or data collection issues better for these surveys.

The training manual provides the foundation for the materials that are to be used in the training workshops. Material will also be prepared for the surveyors that explain the goals of the project and how the data collection efforts have been customized to achieve the project goals.

Conduct Training for Surveyors

There will be a minimum of two trainings sessions. The first training session was undertaken at Itron’s San Diego offices prior to the implementation of the test CMST on-site data collection.

The second training session will provide training for the more in-depth CSS on-site data collection. Implementing the data collection for the CMST prior to the CSS will serve as training for the CSS on those parts of the forms (firmographics, activity areas, and square footage) that overlap.

CSMT/CSS Test Field Training

The initial CMST/CSS training session was undertaken prior to the test implementation of the surveys in December 2011. The test implementation of the surveys focused on Itron survey personnel who have previously completed commercial surveys in California. Limiting the first training session to experienced personnel allowed the team to test the protocols, training information, and forms on survey personnel with the background experience needed to provide constructive input while successfully completing the test sites.

The forms of the test training sessions followed the format that will be used for all future CSS/CMST trainings. The training incorporated an office based training workshop, field surveys implemented at actual sites, and an office based quality control session to review the collected data. The training began with a one-day training workshop, conducted in Itron’s San Diego office for survey staff participating in the field data collection. The training workshop introduced the field staff to the CMST and CSS survey goals and survey forms. The workshop reviewed the training materials. The training instructors included the project manager, a senior engineer, and the field project manager. The project manager provided surveyors with an understanding of why each element of the data to be collected is important while the field manager carefully reviewed the forms and the protocols for filling in the forms.

Draft CSS & CSMT Research Plan

Itron, Inc. 5-8 Survey Development and Implementation

Following the in-office training, the surveyors went in groups to sites to implement the data collection procedure using actual sites. The CMST/CSS sites chosen for training were very small sites that incorporated the full spectrum of equipment needing to be verified under the CMST survey. Following data collection, the team spent the following day in Itron’s San Diego office reviewing the data collected and working to ensure that all surveyors fully understand the forms, the data required, and the appropriate protocol for entering the data on the forms.

After the test training sites, surveyors were given three to eight small sites to complete. These initial sites will be completed, the data quality checked, and any issues reviewed and resolved with the surveyor. Once the issues are reviewed and resolved, the forms will be updated and submitted for data entry. All of the initial sites must be completed to the satisfaction of the field staff manager and the field staff manager must be convinced that the surveyor has a grasp of the data necessary for the survey before the surveyor will be given additional sites. If a surveyor had QC issues with the first set of assigned sites, the extensive review will be repeated with their second set of assigned sites. If quality issues continue, the surveyor will be discontinued from the project. This extensive review and evaluation prior to sending the surveyor to numerous sites will help to ensure that the surveyor understands the data collection needs and how the form is to be completed.

CMST/CSS Field Training

The CMST/CSS training session will incorporate an introduction to the survey goals and the survey forms, visits to sites within multiple segments to collect field data, and a final day in the office to review and quality check the collected information.

The session will start with a one- or two-day training workshop conducted in Itron’s San Diego office for all survey staff prior to the commencement of the fieldwork. A similar training workshop may also be conducted out of Itron’s Oakland office. The training workshop will be conducted using the training materials previously developed. The training materials will be updated to incorporate lessons learned from the initial test training session. The training instructor will be the senior engineer and the field project manager. The training session will address the overall project purpose and scope and the data collection protocols. The Itron field manager will provide the surveyors with an understanding of why each specific element of data to be collected is important. Understanding the importance of the data fields is critical to insuring that all fields are given sufficient care during the data collection effort. The survey personnel will also be given the procedures handbook and will be told to read the handbook and use it as a field reference.

Following the in-office training, the surveyors will go in groups with the Itron field manager and the additional trainers to several sites to review the data collection procedure using actual sites.

The sites chosen for training will cover a full spectrum of equipment and segments. Typical sites

that might be included in the on-site training session include an office or retail building, a restaurant or grocery store, and a school.

After the on-site data collection, the surveyors will return to the office for a one-day review of the data collected and to complete the survey forms. The review session will briefly review each surveyed site, discuss issues or problems encountered at the site, and analyze the data entered on the forms, insuring that the data are correctly entered and that data fields are not overlooked.

The office review session will help the surveyors to learn from the experiences of others and correct any misconceptions that may have existed.

After the test training sites, surveyors were given three to five small sites to complete. These initial sites will be completed, the data quality checked, and any issues reviewed and resolved with the surveyor. Once the issues are reviewed and resolved, the forms will be updated and submitted for data entry. All of the initial sites must be completed to the satisfaction of the field staff manager and the field staff manager must be convinced that the surveyor has a grasp of the data necessary for the survey before the surveyor will be given additional sites. If a surveyor had QC issues with the first set of assigned sites, the extensive review will be repeated with their second set of assigned sites. If quality issues continue, the surveyor will be discontinued from the project. This extensive review and evaluation prior to sending the surveyor to numerous sites will help to ensure that the surveyor understands the data collection needs and how the form is to be completed.

Ongoing Quality Control

Surveyors will be provided with forms representing four to eight CSS/CMST sites to be completed. The sites allocated to a surveyor will be tracked in the on-site tracking workbook.

This workbook identifies which surveyor was assigned the site, if the survey has been completed, which office engineer is QCing the site, and if the site passed the quality checking process and is at data entry or if the site was returned to the surveyor for updating. Using the tracking workbook it will be possible to quickly determine the number of sites completed, the status of various sites, and who is currently responsible for the site’s status. It will also be possible to identify surveyors with reoccurring quality control issues. Surveyors with reoccurring quality control issues will be given the training and oversight necessary to improve their performance. If the survey quality does not improve, these surveyors will be removed from the project.

Surveyors will not be given additional sites unless their pre-assigned sites are completed and turned into the office engineers for quality control review. The on-site surveyors will be expected to turn in their forms on a semi-weekly to weekly basis. The office-based engineers will be reviewing the forms, either forwarding the forms for data entry or returning the forms to the surveyors with questions. Surveyors will be required to update their forms that need review on a weekly basis.

Draft CSS & CSMT Research Plan

Itron, Inc. 5-10 Survey Development and Implementation

The on-site data will be data entered following the quality control process. The data entry personnel will be instructed to report data that does not conform to the field specifications.

These forms will be returned to the office engineers and potentially the surveyors for review and updating.

The data in the on-site database will be analyzed on an ongoing basis. The analysis will also help with quality control. Queries will be established to calculate whole-building energy intensity, lighting power density, and HVAC tonnage per square foot for each site. Sites with measurements outside established ranges will be reviewed to ensure the site’s data was correctly recorded on the form and entered into the on-site database.

6