APPENDIX D: E VALUATION T OOL R EVIEWER I NFORMATION

Dr. David McEntire – Dr. McEntire is an associate professor in the Department of Public Administration at the University of North Texas. He teaches emergency management and his research includes emergency management theory, international disasters, community

preparedness, response coordination, homeland security, and vulnerability reduction. He has received grants funded by the Natural Hazards Center at the University of Colorado, the National Science Foundation, and FEMA in Arkansas and Oklahoma. He is the author of five books and numerous articles that have appeared in emergency management journals. Ellis Stanley – Mr. Stanley has over 30 years of work experience in emergency management. He is currently the Director of Western Emergency Management Services for Dewberry, LLC. Mr. Stanley is known for his work as the General Manager of the City of Los Angeles’ Emergency Preparedness Department, Director of Emergency Management for Brunswick County, North Carolina and later Durham, North Carolina. He was also the Director of the Atlanta-Fulton County Emergency Management Agency during the 1996 Olympics and the Director of

Democratic National Convention Planning for the City and County of Denver Colorado in 2008. Mr. Stanley is also an active member in the emergency management community. He sits on multiple boards including the National Science Foundation and has been the President of the International Association of Emergency Managers.

Sandy Sanderson – Mr. Sanderson is the Emergency Management Coordinator for Dare County, North Carolina, who has over 20 years in emergency management and law enforcement in one of the most hurricane at-risk local communities in the country. Mr. Sanderson has served as a consultant to FEMA, presented at numerous regional and national hazards conferences, and is a former Navy Seal.

Scott Wells – During his career at the Department of Homeland Security, Scott Wells served as a Federal Coordinating Officer for approximately 25 disasters, including hurricanes Katrina and Rita in 2005 and the Columbia Space Shuttle recovery operation in 2003. Prior to his career at DHS, Mr. Wells served as an Army officer for more than 20 years, with 10 years’ Pentagon experience at both the Secretariat and Staff level. He provided Department of Defense (DoD) consequence management support to domestic operations such as the 1996 Olympics, the Oklahoma City bombing in 1995, the 1994 Northridge Earthquake, two Presidential inaugurals, numerous floods, fires and smaller hurricanes, and classified terrorist incidents. Throughout his career he has assisted in the development of DHS doctrine and educational courses in response planning.

123 REVIEWER FEEDBACK FORMS

PURPOSE OF EVALUATION TOOL AND EXPERT FEEDBACK

The purpose of developing this evaluation tool is to assess the quality of local government emergency response plans. The tool is intended to align scholarly research, federal government guidance documents, and local government response plans to better inform:

 practitioners and local government officials in developing or improving emergency response plans;

 scholars in studying the quality of response plans across jurisdictions;

 state and federal emergency management officials in providing plan development guidance and training for local governments; and

 policymakers at the local, state, and federal level to develop policies in emergency management response planning.

Based on your feedback, I will develop an improved evaluation tool. I ask that you provide honest, constructive feedback and evaluate the tool based on your expertise in the field of emergency management.

RESEARCH QUESTION AND METHODOLOGY

My research will answer the question: What constitutes a high quality local government emergency management response plan?

The methodology consists of three parts. First, I developed an evaluation tool that measures plan quality in local government emergency management response plans. The measurement tool is framed using the plan quality principles derived from standards used in planning practice and has been developed through a synthesis of response planning literature, government documents, and local government emergency management response plans. Second, after development of the tool, I tested it on three North Carolina local government response plans and improved it based on that experience. Lastly, I have asked three experts in the field to evaluate the tool and improvements will be made based on their feedback.

124

REVIEWER FEEDBACK QUESTIONS

The following questions will help guide your comments. Any other recommendations are welcome.

1. Is this evaluation tool a useful and accurate way to measure the quality of local government emergency response plans?

2. Does the alignment of planning principles work for emergency response plans? Are there principles that should be omitted or are there other principles that could be added?

3. Are there indicators that are irrelevant to measuring the quality of emergency response plans? Are there redundancies that could be omitted? Is terminology used

appropriately?

4. What is the tool missing? Where are the gaps or weaknesses in the tool? 5. Other comments

DAVID MCENTIRE -REVIEWER FEEDBACK QUESTIONS

The following questions will help guide your comments. Any other recommendations are welcome.

1. Is this evaluation tool a useful and accurate way to measure the quality of local government emergency response plans?

It is always very difficult to measure if an EOP is complete, promising, effective, etc. For instance, one city may have a wonderful plan, but no capabilities to implement it (New Orleans in Katrina comes to mind). Another may lack a plan, but have such a strong sense of cooperation that it is better off than another (Perhaps a city in Japan would be an example here). In addition, it is

sometimes difficult to fully assess a situation because some cities have mitigation and recovery plans, while others only have a response plan. Therefore, any single assessment should be taken with a grain of salt.

That being said, I like what you have put together. It is fairly complete and it would give a general overall assessment of the potential disaster goals and operational capabilities of a community.

2. Does the alignment of planning principles work for emergency response plans? Are there principles that should be omitted or are there other principles that could be added?

Please send me the list of principles (in a single document) so I can comment on them further.

There are many ways to organize response plans and evaluate their potential effectiveness. However, I like what you have put together. While there are always different ways to organize material, I believe you have a fairly complete document. This is the most important goal in such an evaluation.

3. Are there indicators that are irrelevant to measuring the quality of emergency response plans? Are there redundancies that could be omitted? Is terminology used appropriately?

I tend to be of the mindset that it is better to over analyze and assess, rather than be incomplete. Therefore, I do not think anything you have listed is irrelevant.

Some of the lists of organizations might be a little redundant (as they appear in a few places in the plan). The terminology you use is appropriate to the profession and is standard.

4. What is the tool missing? Where are the gaps or weaknesses in the tool?

2.1.3 You may want to include a comment about the industry in the area.

2.2.4 and 2.2.8 may overlap a little (e.g., sewage). They could also be placed near one another since they are related.

I wonder if your list of hazards should be consistent? For instance, in some cases you include causes and impacts and magnitude, and in others you do not.

What about including “mass shooting” as a hazard (e.g., Virginia Tech)?

Should a general operating budget be included in the assessment when determining government capabilities?

Should you include related community organizations as participants in 2.4.1.5? (e.g, churches, CERT teams, Ham Radio clubs, etc.)

I wonder if 2.4.1.6 has some items that should belong elsewhere (traffic, evacuation, communications)? (See section 4.2). Should some of the sections be located together or combined (e.g., evacuation, evacuation of inmates, animal and pet evacuation and sheltering, etc.)?

Should there be mention of redundancies in jurisdictional agency and leader? 4.6 could be labeled as damage and needs assessment.

4.8 could include other environmental issues (debris removal; beach erosion, etc.)

Should faith based orgs be included in 5.2.1?

7.1.1 could have back up redundancy of lead orgs and positions.

7.3.1.1 Some people may also list a functional plan or equate that to an all hazards plan. 5. Other comments

I think you have a great document that is broad and inclusive of all of the issues pertinent to response planning.

Because emergency operations plans are often repetitive, there is difficulty in knowing the best way to organize (and simplify) an evaluation tool. I am curious to know if you thought about different ways to organize the questions. For instance, were there other possible ways to organize this beyond the “principles” approach you took? (I don’t know if there are any other approaches or what the advantages and disadvantages might be. I’m just curious if you thought of any other approaches).

Notes from Scott Wells

- Biggest issue: very comprehensive. Sometimes the strength is the weakness

- Issue: weighting way out of proportion, ex. Public warning system – 0-1-2 versus pandemic threat 0-1. Public warning system 10,000x more important than having pandemic threat - Focus on key things and leave trivial things out

- Look at the whole approach, this checklist does not allow you to do that. It looks at individual tasks

- More subjectivity

- Things to add: concept of operations – mission essential tasks

- **Think about the scoring and how to define it. And figure out analysis – how to analyze** and look at research question

- Should be focusing on war stopper issues – critical issues – and then another list of other not as important issues

- Essence of planning is determining what you need and then what you going to get that you don’t have.

- Capability assessment – required versus available - Make decision on land use versus response principles

- Divide it into three categories – what is critical (required capabilities or resources)

- This is what is critical: Good risk assessment – did it include consequences, it is reasonable and comprehensive, did that risk assessment lead to required capabilities, do they have the required capabilities, how do they make up the shortfalls?

144