Interpretive Evaluation Primer

(1)

Interpretive Evaluation Primer

Phil Waltz, South Platte Park

Whether looking at personal or self-guided programming the keys are: 1. knowing what information you’ll elicit and generate

2. and how you’ll use that data to make improvements. Kinds of Evaluation

Front end – at the beginning of a project to establish some initial

parameters or to gain some initial feedback (audience knowledge base, parameters or to gain some initial feedback (audience knowledge base, expectations, desires, etc.)

Formative – “testing” phase during program or exhibit development (do exhibit instructions work, people gleaning desired information

Summative – how well is the final product working, measured against initial objectives, often critical for grant funded programs.

(2)

Evaluation tools:

Interviews and surveys Interviews and surveys:

•face-to-face

•phone

•mail / email /web (Survey Monkey!)

Visitor observation:

tracking and timing studies (look at

total time, number of stops, types of interaction or

engagement with exhibit)

Focus groups:

Facilitated discussion with limited

appropriate participants from members of a target

audience.

Resource analyses

: Altered impacts on resources? Limits

of Acceptable Change?

(3)

Using evaluation data

:

• tweak materials

during developmetn

(ideally) or perhaps post-development so

they function as desired

•attract desired or maybe

new audience

?

•attract desired or maybe

new audience

?

• improve

future products and efforts

•attract

funders & grants

(4)

Front End Evaluation Results:

Evaluation

Evaluation prior toprior to renovation planning for the visitor center at the renovation planning for the visitor center at the Hatfield Marine Sciences Center (

Hatfield Marine Sciences Center (HMSCHMSC). Evaluations are expected to ). Evaluations are expected to aid the development of a communications plan and selection of

aid the development of a communications plan and selection of

concepts for visitors' interpretive experience. data were collected using concepts for visitors' interpretive experience. data were collected using both a questionnaire and face to face semi

both a questionnaire and face to face semi--structured interviews. structured interviews. Major Major results of the evaluation covered:

results of the evaluation covered: 1.

1. reasons for attending the reasons for attending the HMSCHMSC; ; 1.

1. reasons for attending the reasons for attending the HMSCHMSC; ; 2.

2. visitor expectations;visitor expectations; 3.

3. visitors’ visitors’ knowledge of general science and of marine life and environments; knowledge of general science and of marine life and environments; 4.

4. visitors' level of interest and attitudes toward exhibit themes; visitors' level of interest and attitudes toward exhibit themes; 5.

5. issue areas of greatest interest; Visitors to he issue areas of greatest interest; Visitors to he HMSCHMSC had a strong orientation had a strong orientation toward seeing and closely interacting with marine life and environments.

(5)

Formative Evaluation Results: Reactions to

Prototypes of Selected Exhibits for the

Interpretive Center at the African Burial Ground

What would have made it a “10” for you?

30% more information/ history/biographies30% more information/ history/biographies

13% less reading and more hands13% less reading and more hands--onon

12% seeing it completed12% seeing it completed

11% video / graphics11% video / graphics

11% artifacts / bones/ documents11% artifacts / bones/ documents

8% more space/ more exhibits8% more space/ more exhibits

8% more photographs8% more photographs

7% more for kids7% more for kids

7% more historical context7% more historical context

5% audio5% audio

5% more about the research / discovery5% more about the research / discovery

11% other11% other

(6)

Summative interpretive

evaluation results

To what extent did the new signage at the Brooklyn

Botanic Garden succeed in providing the desired

outcomes? Specifically:

1.

1. Do visitors notice and read the new signs?

Do visitors notice and read the new signs?

2.

2. Can they remember something from the text?

Can they remember something from the text?

2.

2. Can they remember something from the text?

Can they remember something from the text?

3.

3. Do they find the signs engaging?

Do they find the signs engaging?

4.

4. Do they come away with some sense of the overarching

Do they come away with some sense of the overarching

theme, “Plants are essential to life”?

5.

5. Does the signage convey the subtopics and their

Does the signage convey the subtopics and their

relevance to tropical plants?

(7)

Summative evaluation results

1.Overall Appeal: About how many signs did you read?

Frequency N=72 Percent

All or most of them 24 33

About half 25 35

2.Do visitors perceive a central message in the Tropical

Pavilion, and if so, what is it?

Frequency N=50 Percent*

Global dependence on the tropics 21 42

Origin of products we use 18 36

Conservation, preservation 13 26

Interdependence of life within the tropics 7 14 About the rainforest, tropical plants (general) 9 18

(8)

Interpretive Evaluation

Resources and References

Exhibit

Exhibit Labels: An Interpretive ApproachLabels: An Interpretive Approach, by , by Beverly Beverly SerrellSerrell, 1996., 1996.

Introduction

Introduction to Museum Evaluationto Museum Evaluation, by , by MindaMinda BorunBorun and and Randi Randi KornKorn, , Editors,Roxana

Editors,Roxana Adams, Series Editor, published by the Committee on Adams, Series Editor, published by the Committee on Audience

Audience Research and Research and Evaluation of the American Association of Evaluation of the American Association of Museums, Professional

Museums, Professional Practice SeriesPractice Series, 1999., 1999.

Questioning

Questioning Assumptions: An Introduction to FrontAssumptions: An Introduction to Front--End Studies in End Studies in Questioning

Questioning Assumptions: An Introduction to FrontAssumptions: An Introduction to Front--End Studies in End Studies in Museums

Museums, , by by Lynn Lynn D. D. DierkingDierking and Wendy Pollock, 1998.and Wendy Pollock, 1998.

Try

Try It! Improving Exhibits through Formative EvaluationIt! Improving Exhibits through Formative Evaluation, Samuel Taylor, , Samuel Taylor, editor,

editor, and and Beverly Beverly SerrellSerrell, assistant editor, 1991., assistant editor, 1991.

The

The Effectiveness of Effectiveness of NonpersonalNonpersonal Media Used in Interpretation and Media Used in Interpretation and Informal

Informal

Education: An Annotated Bibliography.

Education: An Annotated Bibliography. Wells, Marcella and Lisa Smith. Wells, Marcella and Lisa Smith.

National Association

(9)

Interpretive Evaluation

Resources and References

NPS Social Science Program:

NPS Social Science Program: http://www.nature.nps.gov/socialscience/http://www.nature.nps.gov/socialscience/

Parks

Parks Studies Unit at the University of Idaho Studies Unit at the University of Idaho http://www.psu.uidaho.edu/http://www.psu.uidaho.edu/

((good focus group info!)good focus group info!) These

These professional organizations include exhibit evaluation foci:professional organizations include exhibit evaluation foci: These

These professional organizations include exhibit evaluation foci:professional organizations include exhibit evaluation foci: American

American Association of Museums (Association of Museums (AAMAAM) ) http://www.aamhttp://www.aam--us.org/us.org/

Good annual conference, have offered free registration with Good annual conference, have offered free registration with

commensurate volunteer time! commensurate volunteer time! Visitor