• No results found

Collecting Evaluation Data: End-of-Session Questionnaires

N/A
N/A
Protected

Academic year: 2021

Share "Collecting Evaluation Data: End-of-Session Questionnaires"

Copied!
71
0
0

Loading.... (view fulltext now)

Full text

(1)

Collecting Evaluation Data:

End-of-Session Questionnaires

Ellen Ellen Ellen Ellen

Ellen TTTTTaaaaaylor-Pylor-Pylor-Poylor-Pylor-Poooowwwwwellellellellell Marcus Renner Marcus Renner Marcus Renner Marcus Renner Marcus Renner

Program Development and Evaluation

University of Wisconsin-Extension Cooperative Extension Madison, Wisconsin September 2000 University of Wisconsin-Extension

(2)

We would like to acknowledge all the many useful resources available on post-session evaluations which we used as we developed this manual. Some are noted in the references. We also collected and consulted many instruments from a wide variety of programs and courses--too many to recognize, but all extremely helpful. We’ve included and/or adapted questions from many of these as sample questions. A special thanks goes to Joan Palmer for her patience and skill in formatting and publishing the final manual.

(3)

TABLE OF CONTENTS

Alternatives to End-of-Session Questionnaires

4

Planning Your End-of-Session Questionnaire

5

Suggestions and Tips

7

Kinds of Information: What Do You Want to Know?

10

1. Participant Reactions

12

A. Reaction to the Session

12

B. Reaction to Administration, Facilities and Logistics

19

2. Teaching and Facilitation

22

3. Outcomes

26

A. Overall Benefits

27

B. Commitment to Action

28

C. Perceived Changes in Motivation, Confidence

or Abilities

29

D. Perceived Changes in Knowledge and Skills

30

E. Perceived Changes in Attitudes, Beliefs and Opinions

32

F. Intended Changes in Behaviors and Practices

33

G. Before and After Differences

35

4. Future Programming

37

A. Needs Related to the Current Session

37

B. Preferences for Future Sessions

38

C. Other Planning Information

38

5. Participant Background

41

A. Personal Information

42

B. Additional Information

43

Bibliography

46

Appendix A.

Retrospective Evaluation

The Post-Then-Pre Design

47

(4)
(5)

E

nd-of-session questionnaires are commonly used to collect immediate feedback about educational events including workshops, training, conferences, field days or meetings. We are usually anxious to know how people reacted to our effort and what they learned. Because the participants are in one place, we can get information from them quickly and easily. This information can help us improve our programs and demonstrate results.

Collecting end-of-session feedback is important for various reasons:

! It helps us know immediately what worked and what didn’t.

! It provides information for improving the current program and designing future

programs.

! It provides information helpful in accountability reporting.

! It shows participants and stakeholders that we are interested in improving and that we

value their input.

This manual provides a variety of sample questions and ways to format questions to help you develop your own end-of-session questionnaire. It will help you determine your specific information needs and write questions to answer those information needs. This manual does

not include every possible question. It does not present a single form to use. Nor, does it

identify the best questions to ask. That will depend upon your event, audience and informa-tion needs. Be selective, use what seems appropriate and/or create your own quesinforma-tions to meet you particular needs.

This manual is organized according to the five types of information that can be gained at the end of an educational event. These include information about:

1. Participant reactions 2. Teaching and facilitation 3. Outcomes

4. Future programming 5. Participant background

(6)

Inappropriate for young children, populations with low literacy

Participants are often tired and in a hurry to leave

Potential for positive bias since participants tend to respond to please the presenter

Self-reports

Requires careful question construction to obtain useful information

Advantages Disadvantages

Allows people to remain anonymous

Provides written documentation of people’s own perspectives

Can reach a large number of people

Convenient

Low cost

ALTERNATIVES TO END-OF-SESSION QUESTIONNAIRES

Using a questionnaire to gain evaluative feedback at the end of a session is a very common practice. People, however, may be tired of filling out questionnaires, or another method might be more appropriate given your purpose or participant characteristics. A questionnaire requires that the participants be able to read and write. This may be difficult for young children or for participants with low literacy or disabilities. There are a variety of ways to collect immediate feedback other than a written questionnaire. Consider what will most likely yield the results you seek. Some alternatives include:

! Observations of interactions, skill development or learning1

! A group discussion at the end of the session guided by specific evaluation questions

! Personal interviews with participants

! Photographs or a videotape of the session or particular aspects of the event

Using an end-of-session questionnaire is a fast and easy way to collect data from a large number of people. As with any method it has certain advantages and disadvantages.

(7)

PLANNING YOUR END- OF- SESSION QUESTIONNAIRE

If you’ve decided to use an end-of-session questionnaire, do some upfront planning to ensure more useful results. Start with the planning steps for any evaluation (for further help see Extension Publication, Planning a Program Evaluation (G3658-1). Although these steps are ordered below, many should be thought about simultaneously.

1. Why are you doing an end-of-session evaluation?

Often, we pass out end-of-session questionnaires without much thought or real use in mind. What is your purpose and what do you hope to accomplish by asking participants for their feedback?

2. Who will use the information? How will they use it?

Is this evaluation your idea or is someone else asking you to evaluate? How will the information from the evaluation be used?

3. What do you (and others) want to know?

Take a few moments and write down three to five topics or questions that represent what you want to know about your educational event.

“I would really like to know…”

1. ____________________________________________________________________ 2. _____________________________________________________________________ 3. _____________________________________________________________________ 4. _____________________________________________________________________ 5. _____________________________________________________________________

4. What specific questions will you ask?

Be more specific. Begin to list the questions you and others want answered. Do you want to know about people’s reactions to the presentations, knowledge gains, satisfaction with the facility, what participants expect to do as a result of attending, etc.? Be clear and precise about what you want to know and how you will use the information. Differentiate between what would be nice to know and what you need to know.

5. How should the form be designed and formatted?

How long do you want it to be? How should the questions be ordered? Do you have a graphic or design to make the questionnaire more eye pleasing? Remember, the look of the question-naire is critical.

(8)

6. Who will check the questionnaire?

Allow enough time in your planning to have someone else review your questionnaire. In good evaluation, we always pilot test our instruments and data collection procedures. At the minimum, ask a colleague or interested other to read and critique your questionnaire.

7. When will the form be used/distributed?

Will the form be included in the conference packet, placed on tables, handed out at the end of each subsection, at the end of the whole conference or when? Experience shows that you should allow time for filling out the questionnaire within the designated time frame of the program.

8. How will the data be managed and analyzed?

If you are expecting only a limited number of participants, you may hand tally the results. For larger numbers of participants, you may consider a computer spreadsheet program, data management software or a statistical package for data processing and analysis. Don’t forget to make time for and think about interpretation. Words and numbers do not speak for them-selves. You will want to answer the questions: What did we learn? What are the recommen-dations?

9. How will you use your results for program improvement and external reporting?

Plan for sharing and communicating your results. Participants, key stakeholders, program staff, colleagues, funders and others will be interested in the results. Reporting may be as simple as relating the results at a meeting or preparing a short report or media release.

10. Who is responsible for each task above? How much time will each take?

Will you do it all alone or will you have help? Sharing the tasks of evaluation with others spreads ownership, understanding and commitment and generally results in a better product. Often we put together an end-of-session questionnaire hurriedly and at the last minute. With a little foresight and planning, you can obtain more useful information.

(9)

! Make the evaluation form easy to complete. Keep your questions brief, choose a font

that is easy to read and leave plenty of white space on the form, so it does not appear cluttered.

! Keep your audience in mind. The questionnaire you use will depend on the audience –

age, literacy level, etc. Phrase the questions and format the questionnaire to match the characteristics of your audience.

! Cluster similar items to save space and time. For example:

Rather than asking separate questions: Cluster items into one question:

! Think about how to write the question. There are many different ways to phrase a

question (see page 8). Decide what types of questions provide the information you want. Yes-No response, one best answer choice, rating, ranking, comparison, matching, list or open-ended questions are all possibilities. UWEX Cooperative Extension’s Questionnaire Design: Asking Questions with a Purpose (G3658-2) reviews how to write questions in greater detail. Examples of different rating scales are found in Appendix 2.

! ! ! !

! Consider cultural issues. When developing an end-of-session questionnaire for persons

from different cultures, consider carefully the meaning of the words you use. Concepts may have different meaning even within the same ethnic group. Ensure that the structure of the questions and the response instructions are clear and understandable. To be on the safe side, ask a representative of the cultural community to review your questionnaire. !

! ! !

! Keep the form short. It is important to remember that participants are often tired at the end

of an educational event, so questionnaires need to be focused and concise. Work towards obtaining the most information that can be provided in the least amount of time. Be clear about your purpose and limit data collection to the information you are sure to use. !

! ! !

! Provide time for participants to complete the evaluation forms. To get feedback from

most participants, schedule time for them to fill out the questionnaire as part of your educational event. You may wish to save the final announcements for the event until after participants complete the questionnaire.

! ! ! !

! Guard against a positive bias. Often participants give high marks to please the instructor.

Stress to participants that you are interested in both the strengths and weaknesses of the session. Include a question that asks about weaknesses or ways to improve.

SUGGESTIONS AND TIPS

1. How would you rate each of the following:

poor fair good excellent a. the content _____ _____ _____ _____ b. the facility _____ _____ _____ _____ c. the teaching _____ _____ _____ _____ 1. How would you rate the content?

___ poor ___fair ___good ___excellent 2. How would you rate the facility?

___ poor ___fair ___good ___excellent 3. How would you rate the teaching?

(10)

TYPES OF QUESTIONS

Questions can be open or close-ended. Within close-ended questions, various possibilities exist. Spend some time making sure the wording of your question solicits the information you want.

OPEN-ENDED QUESTION:

What did you find most useful about today’s workshop?

CLOSE-ENDED QUESTIONS:

1. YES – NO question:

Did you find today’s workshop useful? a. ___ No

b. ___ Yes 2. One best answer question:

Of the following topics, which ONE was most useful to you? (Check ONE please.) a. ___ physical punishment b. ___ setting limits c. ___ consequences d. ___ listening e. ___ anger f. ___ challenging behaviors 3. Check all that apply question:

Which of the following topics were most useful to you? (Check all that apply.) a. ___ physical punishment b. ___ setting limits c. ___ consequences d. ___ listening e. ___ anger f. ___ challenging behaviors 4. Rating question

To what extent did you find today’s workshop useful? a. ___ not useful

b. ___ somewhat useful c. ___ very useful d. ___ extremely useful 5. Ranking question

Of the topics we covered today, please rank their usefulness to you in order of 1, 2, 3, 4, 5, 6 where, 1 = most useful and 6 = least useful.

a. ___ physical punishment b. ___ setting limits c. ___ consequences d. ___ listening e. ___ anger f. ___ challenging behaviors

(11)

! ! ! !

! Protect the anonymity of the respondent. Signatures on end-of-session evaluation

forms are usually optional. Forms can be collected on an empty desk or in a box by the door rather than returned directly to you.

! ! ! !

! Use the feedback you receive. Making changes based on the feedback you receive

shows participants that you value their input. Explain how you plan to use the information as you distribute the forms.

! ! ! !

! Proof read, edit and pilot test.

CAUTION

End-of-session evaluation questionnaires can provide valuable information on different aspects of an educational event useful in program planning, program improvement and accountability. Still, recognize them for what they are: self-reports, providing partici-pants’ perceptions at one point in time – immediately after an event. They do not test participants or verify outcomes or document actual learning or changes in behavior. Participant perceptions provide valuable information, but what people believe they learn and what they actually learn may be different. In fact, there is no definite relation-ship between what people feel they learn and what they actually learn (Dixon, 1990; LeRouzic and Cusick, 1998). Consequently, exercise caution when using end-of-session questionnaires to document change. Think about combining end-of-end-of-session evaluation forms with other types of data collection to provide a more complete accounting of your educational event and its benefits.

(12)

KINDS OF INFORMATION: WHAT DO YOU WANT TO KNOW?

In the past we often used end-of-session questionnaires as a “happiness index” to find out how satisfied people were with the facility, the instruction, the content – how much people liked us and our program. Increasingly, we see that with careful thought about what we’d like to know and specific wording of our questions, we can collect more useful information for program improvement and accountability purposes.

There are five different kinds of information that end-of-session questionnaires can provide. Do you want to know about:

1. Participant Reactions (pages 12-21)

How do participants feel about the program? What did they like? What didn’t they like? Participant reactions are a measure of satisfaction. They tell you how pleased people are, if they are likely to attend a future session or if they are likely to recommend the session to others. Reactions are not outcomes. They do not tell you what people learned or how people have benefited. But, positive outcomes are more likely if people react positively and are satisfied.

2. Teaching and Facilitation (pages 22-25)

How you are viewed as a teacher or facilitator is often a high priority. Or, you may have invited others to teach or facilitate the session. Participant feedback can help you identify strengths, weaknesses and areas for improvement. This is how you develop professionally as an educator. It also helps you determine what you will do differently next time.

3. Outcomes (pages 26-36)

What are the results of the educational event? What are the benefits for the participants and others? While you may not be able to actually measure knowledge gains, skill development, changes in attitudes, motivations or opinions, you can ask questions about what people think they gained and intend to do as a result of the educational effort. There also may be unin-tended or unexpected outcomes you’ll want to ask about. Be sure that the outcomes you ask about are possible given the scope and length of the educational event.

4. Future Programming (pages 37-40)

At the end of a session is an opportune time to ask participants about any additional educa-tional needs or interests that they might have. This information can help you plan future programs or redirect current efforts.

5. Participant Background (pages 41-45)

You may be interested in gathering information about participants’ backgrounds to determine who is attending and who you may be missing. Knowing relevant characteristics of the

(13)

participants (such as their age, skill level, prior knowledge) can also help you understand their other responses on the questionnaire.

This manual provides sample questions for each of these five kinds of information. Typically end-of-session questionnaires include questions from several of these areas. The sample questions that follow are not meant to be complete, perfect or exhaustive. Change them or add others to meet your particular information needs.

(14)

1. PARTICIPANT REACTIONS

End-of-session questionnaires generally ask about participant’s reactions – what people thought about the program, including content, materials, facilities and administration. Were participants pleased with the session? Participant reaction is important to consider when redesigning or continuing your educational effort. Reactions do not represent outcomes but they are a major input for improving programs. Your questions might focus on people’s reactions relative to the session’s value, usefulness, quality, applicability, appropriateness, relevance or how well it met expectations. The following lists indicate a variety of reactions that might be of interest. Reactions to the presenters are covered in a separate section, Teaching and Facilitation.

A. Reactions to the session including its content and delivery such as its

! Goals and objectives

! Content and information provided

! Learning environment

! Activities

! Components of the event

! Choice of sessions

! Materials

! Exhibits, demonstrations

! Teaching methods

! Instructional aids

B. Reactions to the administration, facilities and logistics

! Pre-event information

! Registration

! Communications

! Length, location, time

! Food

! Costs

A. Reactions to the Session

1. How would you rate the conference overall? (Circle one.) 1 = Unsatisfactory 2 = OK 3 = Good 4 = Excellent

2. To what extent was attending this training worth your time? (Circle one.)

Not at All Great Deal

1 2 3 4 5 6

(15)

4. Overall, was this meeting useful to you? a. Yes

b. No OR

Overall, to what extent was this meeting useful to you? (Circle one.)

Not Useful Very Useful

1 2 3 4 5 6

5. To what extent do you think you can apply the information presented today to your work? (Circle one.)

Not at All A Great Deal 1 2 3 4 5 6

6. Of the information presented today, how much is usable to you? (Check one.) a. ___ 0-20% b. ___ 21-40% c. ___ 41-60% d. ___ 61-80% e. ___ 81-100% a. Yes b. No c. It depends Please explain: __________________________________________

3. Please rate the workshop on the following items: (Circle one number for each item.)

a. Content useful 1 2 3 4 5 of little use b. Organization excellent 1 2 3 4 5 poor

c. Use of instructional aids appropriate 1 2 3 4 5 inappropriate d. Creating interest in topic stimulating 1 2 3 4 5 boring e. Involvement of participants excellent 1 2 3 4 5 poor

f. Pace of delivery appropriate 1 2 3 4 5 inappropriate g. Workbook materials helpful 1 2 3 4 5 of little help

(16)

8. Would you recommend this satellite program to others? (Circle one.) a. Yes

b. No c. Uncertain OR

Definitely Yes Definitely No 1 2 3 4 5 6

9. What would you estimate as the dollar value of this conference? (Circle one.) a. Less than $25 b. $26 - $50 c. $51 - $75 d. $76 - $100 e. $101 - $300 f. $301 - $500

g. Other, please specify: ___________________________________________

10.The meeting has me feeling (Circle one number for each item.):

a. bored 1 2 3 4 stimulated

b. dissatisfied 1 2 3 4 satisfied

c. not motivated 1 2 3 4 motivated

d. not focused 1 2 3 4 focused

11.For my experience level, the workshop was: a. ____ too basic

b. ____ about right c. ____ too advanced

12. Overall the clinic:

a. ____ was a good learning experience b. ____ was a mediocre learning experience

c. ____ was of little value and should NOT be continued

a. ____ too easy b. ____ just right c. ____ too difficult a. ____ too elementary b. ____ about right c. ____ too technical a. ____ too simple b. ____ just right c. ____ too complicated OR OR OR OR

(17)

13. Please evaluate this session by circling a number on the following scale: (Circle one number for each.)

Poor Excellent a. general overall impression 1 2 3 4 5 b. usefulness of subject matter for me

professionally 1 2 3 4 5

c. usefulness of subject matter for me

personally 1 2 3 4 5

14. What was most useful about the demonstration?

15. What was least useful about the demonstration?

16. What I found most useful was:

17. What three things from this conference were most helpful to you? a. ___________________________________

b. ___________________________________ c. ___________________________________

18. What was the single best thing about the day?

19. What were the weaknesses of this workshop?

20. What were the strong points of this training?

21. Please comment on the quality of this workshop:

22. The things that could have been omitted from the day are:

23. What I was looking for from this workshop but didn’t get:

24. Have you attended similar workshops provided by others? (Circle one.) a. Yes

b. No

(18)

25. Were you satisfied with the field day? (Circle one.) a. Yes

b. No

Why or why not?

26. How can the conference be improved?

27. What would make the workshop more useful?

28. My general comments and suggestions for improving this conference are:

29. What part of the conference did you like most?

30. What part of the conference did you like least?

31. Six presentations were made at today’s seminar. Circle the response that best reflects how well you felt each presentation met your needs.

Not Somewhat Very Did Not [List title of presentations.] Well Well Attend a. ___________________________ 1 2 3 9 b. ___________________________ 1 2 3 9 c. ___________________________ 1 2 3 9 d. ___________________________ 1 2 3 9 e. ___________________________ 1 2 3 9 f. ___________________________ 1 2 3 9

32. Please evaluate the presentations made today. (Circle one number for each.)

[List title of presentations.] Poor Excellent a. ___________________________ 1 2 3 4 5 6 7 8 9 10 b. ___________________________ 1 2 3 4 5 6 7 8 9 10 c. ___________________________ 1 2 3 4 5 6 7 8 9 10 d. ___________________________ 1 2 3 4 5 6 7 8 9 10

(19)

33. Please give your reaction to the following sessions:

[Title of session] Poor Fair Good Excellent Not Applicable

a. Presentation 1 2 3 4 NA

b. Usefulness to my work 1 2 3 4 NA

c. Overall rating 1 2 3 4 NA

d. Comments:

[Title of session] Poor Fair Good Excellent Not Applicable

a. Presentation 1 2 3 4 NA

b. Usefulness to my work 1 2 3 4 NA

c. Overall Rating 1 2 3 4 NA

d. Comments:

34. How valuable did you find each exhibit?

[Title of exhibits] Not at All Very Didn’t Valuable Valuable Visit

a. _____________________ 1 2 3 4 5 NA

b. _____________________ 1 2 3 4 5 NA

c. _____________________ 1 2 3 4 5 NA

35. From the following list of topics that were covered today, please indicate how useful you found each to be. (Please circle one number for each topic.) [List topics.] Not Slightly Moderately Very

Useful Useful Useful Useful

a. ____________________________ 1 2 3 4

b. ____________________________ 1 2 3 4

c. ____________________________ 1 2 3 4

d. ____________________________ 1 2 3 4

36. How well did the information presented meet your expectations? (Please circle one number.)

A Little A Great Deal

1 2 3 4 5 6

(20)

38. Please rate the workshop content on the following. (Place an X in the appropriate space.)

Very Very

Poor Fair Good Good Excellent

a. Met my needs ___ ___ ___ ___ ___

b. Suitable to my level of experience ___ ___ ___ ___ ___

c. Interesting ___ ___ ___ ___ ___

d. Up-to-date and forward-looking ___ ___ ___ ___ ___ e. Consistent with description ___ ___ ___ ___ ___

f. Understandable ___ ___ ___ ___ ___

g. Logically organized ___ ___ ___ ___ ___

h. Included sufficient examples ___ ___ ___ ___ ___ i. Overall rating of workshop content ___ ___ ___ ___ ___

39. Please indicate your agreement with the following statements about the seminar by circling your answer. 1=Strongly Disagree; 2=Disagree; 3=Neither Agree nor Disagree; 4=Agree; 5=Strongly Agree

Strongly Strongly Disagree Agree The educational materials and content…

a. Helped me better understand the issue 1 2 3 4 5 b. Provided information relevant to my work 1 2 3 4 5 c. Were based on current, up-to-date information 1 2 3 4 5 d. Addressed the topic identified in the title 1 2 3 4 5

e. Were well organized 1 2 3 4 5

f. Were easy to understand 1 2 3 4 5

g. Will be of great immediate use to me 1 2 3 4 5

40. How useful do you think the project manual will be?

Of Little Use Of Great Use

1 2 3 4 5

41. Please indicate your evaluation of the workshop.

Poor Fair Good Excellent a. Was organized and followed a logical order 1 2 3 4 b. Provided information I can use in my work 1 2 3 4

c. Will be helpful in my career 1 2 3 4

d. Expanded my thinking about the topic 1 2 3 4 e. Related information to client needs 1 2 3 4 f. Provided material I can use with clients 1 2 3 4

(21)

42. Please rate the workbook on the following:

Poor Fair Good Excellent a. Easy to read

b. Easy to understand

c. Provides information you can use d. Effectively integrates information on

pesticides and microbial contamination

43. What reactions do you have to the methods that were used today? (Check one for

Poor Fair Good Excellent a. Readings selected

b. Lectures

c. Visual aids, transparencies d. Videotapes

e. Group activities f. Group discussion

g. Consultation on problems

44. Please check the TWO topics that were most useful to you? [List topics.]

____ a. ______________________________ ____ b. ______________________________ ____ c. ______________________________ ____ d. ______________________________

B. Reactions to Administration, Facilities and Logistics

You may be interested in people’s reactions to the location of the event, the time, accommo-dations and food, as well other logistical details.

1. Was your stay satisfactory? (Circle one.) a. Yes

b. No

c. Please explain: ____________________ each category.)

(22)

2. Was the time of the meeting convenient? (Circle one.) a. Yes

b. No

c. If no, a more appropriate time would be: _________

3. Was the date [specify the date] of the training convenient? (Circle one.) a. Yes

b. No

4. Was the location of the meeting convenient? (Circle one.) a. Yes

b. No

c. If no, please suggest another location ____________________________________

5. Were you able to get to the meeting location easily? (Circle one.) a. Yes

b. No

6. How far did you travel to get to the meeting? _______________ miles.

7. How would you rate the:

a. Snacks at break Poor Fair Good Excellent

b. Meals Poor Fair Good Excellent

c. Lodging Poor Fair Good Excellent d. Parking Poor Fair Good Excellent f. Service Poor Fair Good Excellent Comments:

8. Do you think the registration fee was: (Circle one.) a. Too high?

b. Just right? c. Too low?

(23)

9. How would you rate the satellite conference on the following? (Circle one number for each.

Low High a. Comfortable setting for learning 1 2 3 4 5

b. Good audio reception 1 2 3 4 5

c. Good video reception 1 2 3 4 5

d. Adequate time for interaction 1 2 3 4 5

10. Circle one answer for each question.

a. Could you see? Yes No

b. Could you hear? Yes No

c. Was the room temperature comfortable? Yes No d. Were the coffee/stretch breaks convenient? Yes No e. Comments:

11. What aspects about the conference logistics (i.e. food, evening activities, room layout) could be improved?

12. How would you rate the following:

Less than Excellent Adequate a. Preprogram communications 1 2 3 4 5 b. Registration 1 2 3 4 5 c. Location of conference 1 2 3 4 5 d. Conference facility 1 2 3 4 5 e. Meeting rooms 1 2 3 4 5 f. Convenient time 1 2 3 4 5

(24)

2. TEACHING AND FACILITATION

You may be interested in your own performance or that of others who are presenting or leading the event. Knowing how well you are teaching (transmitting information) and facili-tating (assisting the learning process or guiding a group) helps you improve professionally and identify changes that may be needed in future programs. Increasingly, Extension educa-tors serve as facilitaeduca-tors of groups rather than teachers per se, so your questions need to be worded appropriately.

There are numerous aspects of teaching and facilitation that you might wish to include in your questions including:

! Overall effectiveness

! Knowledge of subject matter

! Ability to facilitate learning

! Ability to facilitate group process

! Preparation and organization

! Attitude toward learners

! Ability to engage learners

! Use of visuals and teaching materials

! Communications ! Use of time

! Approachability

! Answering questions

! Adjustments to interests, level of participants

1. To what extent did the instructor:

Not Great Much Deal

a. Know the subject 1 2 3 4 5

b. Encourage participation 1 2 3 4 5

c. Answer questions completely 1 2 3 4 5

d. Respect your knowledge and experience 1 2 3 4 5

e. Use appropriate examples 1 2 3 4 5

f. Provide clear explanations 1 2 3 4 5

g. Generate a desire to learn 1 2 3 4 5

h. Use an effective mix of lectures and exercises 1 2 3 4 5

(25)

2. To what extent do you agree or disagree with the following statements:

The instructor . . . Strongly Strongly Disagree Agree

a. Was well prepared 1 2 3 4 5

b. Was interested in helping me learn 1 2 3 4 5

c. Stimulated my desire to learn 1 2 3 4 5

d. Answered questions clearly 1 2 3 4 5

e. Related program content to real-life situations 1 2 3 4 5

f. Gave clear explanations 1 2 3 4 5

g. Held my attention 1 2 3 4 5

h. Presented information that will help me 1 2 3 4 5

3. Please give your frank and honest reactions. (Place one number in each cell where 1=Poor, 2=Fair, 3=Good, 4=Very Good, 5=Excellent.)

Leader Knowledge Presentation Discussion Use of Overall

of Subject Audio Visuals

a. Warren Peace b. Guy Wire c. Natalie Dressed d. Pete Moss

4. Please give your evaluation of the speakers by circling the appropriate letter: (E=Excellent, G=Good, F=Fair, P=Poor)

Speaker Content of Presentation Speaking Ability Course Materials a. Warren Peace b. Guy Wire c. Natalie Dressed d. Pete Moss E G F P E G F P E G F P E G F P E G F P E G F P E G F P E G F P E G F P E G F P E G F P E G F P
(26)

5. Please put an X along the line between the pairs of choices to give your opinion about the instructor.

Instructor a. Muddled Organized b. Boring Interesting c. Inhibiting Motivating d. Discouraging Encouraging e. Unresponsive Responsive

6. Please circle one number that most accurately indicates your evaluation of the presenter.

PREPARATION

1 2 3 4 5 Much planning Little planning

evident evident

KNOWLEDGE OF SUBJECT MATTER

1 2 3 4 5

Meaning always clear Meaning often obscure KEEPING THE SESSION INTERESTING

1 2 3 4 5 Participants actively Considerable

engaged boredom

EXPLANATIONS

1 2 3 4 5

Very clear Confusing

ANSWERING QUESTIONS

1 2 3 4 5 Answers them fully Does not and directly answer them

LEVEL OF PRESENTATION

1 2 3 4 5 Easily followed Too advanced

(27)

7. What do you think are the instructor’s strengths?

8. How could the instructor improve?

9. How well did the agent facilitate the group process? (Circle one.)

Not Well Very Well

1 2 3 4 5

10. To what extent did the educator help the group:

Not Much A Great Deal

a. Build consensus 1 2 3 4 5

b. Establish priorities 1 2 3 4 5

c. Gain necessary information 1 2 3 4 5

d. Listen to each other 1 2 3 4 5

e. Keep to the agenda 1 2 3 4 5

11. Would you recommend this instructor to others?

Yes, No, Suggest You

Definitely Find Someone Else

(28)

3. OUTCOMES

Outcomes are the results of the educational event and the benefits people gain from partici-pating. As you write your questions, consider what type of outcomes you would expect participants to be able to reflect on at the end of the event. Consider the objectives of the event, how it is implemented, the length of the session, what it covers, the experience of program participants, etc. Keep your questions realistic and consistent with the scope, content and audience of the event.

As self-reports, end-of-session evaluation questionnaires reflect people’s perceptions of outcomes. Individual perceptions are real and valuable, but they may or may not reflect actual learning or change.

There are a variety of outcomes you might be interested in, such as ! Overall benefits that participants attribute to the session ! Commitment to action

! Perceived changes in motivation, confidence or abilities ! Perceived changes in knowledge and skills

! Perceived changes in attitudes, beliefs and opinions ! Intended changes

! Perceptions of before and after differences

Sara Steele (1995) cautions that if we think only in terms of new knowledge gains, we may be missing a variety of important outcomes. She offers a list of other knowledge gains and kinds of results your program may be fostering such as:

! Expanded understanding

! Increased insights into what was already known ! Clarified things that had been learned

! Refocused attention to a topic

! Helped integrate information that had been heard ! Helped in understanding one’s own beliefs ! Challenged thinking

! Stimulated interest to learn more ! Stimulated new thinking

! Provided ammunition to use in an argument ! Triggered ideas

! Helped in applying information ! Encouraged action

(29)

A. Overall Benefits

End-of-session questionnaires often include a question about the general benefit or value of the event.

1. What did you gain from this workshop? (Check all that apply.) a. ____ Answers to my questions

b. ____ Resource materials I can use c. ____ Ideas I can try immediately d. ____ Names of other people to contact e. ____ Nothing new

f. Anything else? ________________________________

2. How much of the content of this workshop did you already know? (Check one.) a. Nothing ______

b. A little ______ c. Quite a bit ______ d. Most of it _______

4. Which of the following benefits do you attribute to this clinic?

[List benefits.]

a. _____________________________________ Yes No b. _____________________________________ Yes No c. _____________________________________ Yes No

5. To what extent did the conference make you think?

Not at All A Great Deal

1 2 3 4 5

6. How do you think this workshop will help you in your work?

3. What percentage of the content covered today did you already know? (Check one.) a. ___ 0-25%

b. ___ 26-50% c. ___ 51-75% d. ___ 76-100%

(30)

B. Commitment to Action

People who make a public commitment to action are more apt to follow through than those who do not make such a commitment. One technique is to ask participants to express what they intend to do based on the learning experience before they leave the session. They may simply list one thing they intend to do on a paper or note card. If the instructor keeps a copy, follow-up is possible.

1. What one or two ideas from this workshop will you try? OR

Do you intend to try any of the techniques discussed today? a. Yes ____

b. No ____

c. If Yes, what do you plan to try?

2. How do you plan to use the information from the conference? 7. Indicate one or two main things you learned or gained today.

8. The most important thing I learned today was:

9. What did you gain from this experience that was unexpected?

10. What new things did you learn today?

11. The conference was designed to build skills in [. . . .]. To what extent did the conference help you to: (Circle one number for each item.)

[List specific skills.] Not at All Very Little Somewhat

a. _____________________ 1 2 3 4

b _____________________ 1 2 3 4

c. _____________________ 1 2 3 4

d. _____________________ 1 2 3 4

(31)

3. What will you do differently as a result of this demonstration? OR

Do you intend to do anything differently as a result of today’s workshop? a. Yes ____

b. No ____

c. If Yes, please describe:

4. What one thing will you try as a result of this clinic?

5. What materials or information do you intend to use? [List materials and information.]

a. _____________________ Yes No Not applicable to me b. _____________________ Yes No Not applicable to me c. _____________________ Yes No Not applicable to me

6. Which of the following will you consider doing as a result of this workshop:

[List actions.]

a. _____________________ Yes No Not applicable to me b. _____________________ Yes No Not applicable to me c. _____________________ Yes No Not applicable to me

C. Perceived Changes in Motivation, Confidence or Abilities

1. To what extent do you feel more able to [….] as a result of this training:

Not at All Great Extent

1 2 3 4 5

2. As a result of this course, I feel that I can:

(32)

D. Perceived Changes in Knowledge and Skills

Short quizzes or content specific questions may be used to assess knowledge at the end of the session. This shows what people know at the end of the event. Without a baseline, compari-son or linkage to the event, however, do not attribute this knowledge to the educational event. Participants might have had this knowledge before attending.

1. In what areas do you feel knowledgeable enough to make decisions and in what areas do you feel that you need more information or counsel?

[List topic areas.] Knowledgeable Need More Help a. ___________________ ____________ _____________ b. ___________________ ____________ _____________ c. ___________________ ____________ _____________ d. ___________________ ____________ _____________

3. Below is a list of things that some people have gained from taking part in a work-shop like this. (Please check any which you feel you have gained.)

a. ____ Greater confidence b. ____ Increased morale

c. ____ Support from others with similar problems d. ____ Increased motivation

f. ____ Other __________________________________________

4. Benefit to me from today’s workshop:

Poor Fair Good Excellent NA a. Stimulated me to think ____ ____ ____ ____ ____ b. Motivated me to want to learn more ____ ____ ____ ____ ____ c. Motivated me to do something different ____ ____ ____ ____ ____

5. The opportunity to present what I learned to my peers helped to reinforce my learning and confidence.

Strongly Strongly

Disagree Agree

(33)

2. As a result of this conference, to what extent do you understand the following topics? (Circle one number for each topic.)

Not Understand Understand Already [List topics.] Very Well Most Very Well Knew a. ______________________ 1 2 3 4 b. ______________________ 1 2 3 4 c. ______________________ 1 2 3 4 d. ______________________ 1 2 3 4

3. Please circle the answer that best describes how much you learned today about each topic listed.

[List topics.]

a. ______________________ Nothing A Little Bit Quite a Bit A Lot b. ______________________ Nothing A Little Bit Quite a Bit A Lot c. ______________________ Nothing A Little Bit Quite a Bit A Lot d. ______________________ Nothing A Little Bit Quite a Bit A Lot

4. To what extent do you feel you’ve gained new skills in:

[List skill areas.] Not Much A Lot a. ______________________________ 1 2 3 4 5 b. ______________________________ 1 2 3 4 5 c. ______________________________ 1 2 3 4 5 d. ______________________________ 1 2 3 4 5

5. To what extent has this workshop helped prepare you to do the following: (Circle one number for each skill area.)

No Somewhat Much Better More [List skill areas.] Difference Better Prepared Prepared Confused a. ______________________ 1 2 3 4 b. ______________________ 1 2 3 4 c. ______________________ 1 2 3 4

6. What did you learn at this field day? (Circle one number for each item.)

Gave Me New Reinforced What Nothing [List expected knowledge changes.] Information I Already Knew New Presented a. Ways to prevent groundwater pollution 1 2 3 b. Signs and symptoms of pesticide poisoning 1 2 3

(34)

E. Perceived Changes in Attitudes, Beliefs and Opinions

1. Have you changed your attitude about water consumption in any way as a result of this trial? (Circle one.)

a. Yes b. No

c. If yes, please specify:

2. As a result of today’s session, please indicate your opinion on the following statement. (Circle one number.)

“I wish this community had more outdoor recreation centers.” a. Strongly agree

b. Mildly agree

c. Neither agree nor disagree d. Mildly disagree

e. Strongly disagree

3. As a result of today’s clinic, do you feel that [….] is:

No Change in Riskier Safer Opinion

1 2 3 4 NC

7. As a result of participating in the workshop:

SA = Strongly Agree; A = Agree; N = Neither Agree Nor Disagree; D = Disagree; SD = Strongly Disagree

SA A N D SD a. I learned new skills that will help me in my job ____ ____ ____ ____ ____ b. I increased my knowledge about [….] ____ ____ ____ ____ ____ c. I am more aware of [....] ____ ____ ____ ____ ____ d. I am stimulated to learn more ____ ____ ____ ____ ____

8. To what extent do you think you learned about the following topics:

[List topics.] Little Some A Lot a. ______________________________ 1 2 3 4 5 b. ______________________________ 1 2 3 4 5 c. ______________________________ 1 2 3 4 5 d. ______________________________ 1 2 3 4 5
(35)

4. As a result of today’s workshop, to what extent do you feel that your attitude has changed about:

[List topics.] Not at All Not Much Somewhat A Great Deal a. __________________________ _______ _______ _______ _______ b. __________________________ _______ _______ _______ _______ c. __________________________ _______ _______ _______ _______

5. Are you thinking differently about [….] as a result of today’s workshop? a. Yes

b. No

c. Please comment if you care to:

6. Please indicate how your opinion has changed as a result of participating in this program.

No Less Slightly Moderately Much Change Positive More More More [List practices.] Positive Positive Positive a. ________________________ _______ _______ ________ _______ _______ b. ________________________ _______ _______ ________ _______ _______ c. ________________________ _______ _______ ________ _______ _______ d. ________________________ _______ _______ ________ _______ _______ e. ________________________ _______ _______ ________ _______ _______

F. Intended Changes

Changes in behaviors and practices only become evident once the participant returns to his/ her home, work or life. In an end-of-session evaluation form, you can only ask about changes in behavior and practices that participants intend or plan to make.

1. Do you plan to [list the practice or change expected]? (Circle one.) a. Yes

b. No

c. Don’t know

d. If yes, how? Please specify:

(36)

3. Which of the following practices do you intend to use that you did not use before this training? (Check one answer for each.)

Used Before Intend Don’t Intend [List practices.] to Use to Use a. __________________________ ____ ____ ____ b. __________________________ ____ ____ ____ c. __________________________ ____ ____ ____ d. __________________________ ____ ____ ____ e. __________________________ ____ ____ ____

Currently Expect to Not Sure [List change.] Doing This Make This Change About Doing This a. ______________________ ____ ____ ____ b. ______________________ ____ ____ ____ c. ______________________ ____ ____ ____ d. ______________________ ____ ____ ____ e. ______________________ ____ ____ ____

5. During the course of this meeting, we talked about a variety of ways you can eat to have a healthier diet. Which of the following do you expect to do? (Check those that you intend to do.)

[List dietary changes.]

___ a. ______________________________________________ ___ b. ______________________________________________ ___ c. ______________________________________________ ___ d. ______________________________________________ ___ e. ______________________________________________

4. Please check the appropriate answer for the changes we talked about today.

6. As a result of this training, do you intend to make changes in the following area?

[List areas.] Low High a. _____________________________ _____ _____ _____ _____ b. _____________________________ _____ _____ _____ _____ c. _____________________________ _____ _____ _____ _____ d. _____________________________ _____ _____ _____ _____ e. _____________________________ _____ _____ _____ _____

(37)

G. Perceptions of Before and After Differences

Comparison adds strength to the assessment of outcomes. One way to do this in an end-of-session evaluation is to ask participants to compare their knowledge or ability at the begin-ning of the event with that at the end. This evaluation design, called the retrospective or post-then-pre design, has certain advantages over the traditional pre-post test design (see Rockwell, 1989 and Le Rousic and Cusick, 1998.). It is a popular and valid questionnaire design (see Appendix 1).

1. Please CIRCLE one rating for both BEFORE and AFTER the meeting:

Low Moderate High (1) I understood the value of strategic thinking

a. Before the meeting? 1 2 3 4 5

b. After the meeting? 1 2 3 4 5

(2) I knew how to write a mission statement

a. Before the meeting? 1 2 3 4 5

b. After the meeting? 1 2 3 4 5

(3) I could perform a SWOT analysis

a. Before the meeting? 1 2 3 4 5

b. After the meeting? 1 2 3 4 5

7. As a result of this conference, what changes do you anticipate?

I plan to: Stop or Decrease No Change Start or [List items.] Discontinue Increase a. ________________________ _____ _____ _____ _____ b. ________________________ _____ _____ _____ _____ c. ________________________ _____ _____ _____ _____ d. ________________________ _____ _____ _____ _____ e. ________________________ _____ _____ _____ _____

8. As you think about the next three months, circle the number which best describes the degree to which you WILL ENGAGE in the following practices. (1= Almost

never, 2=Seldom, 3= About half the time, 4=Often, 5=Almost always) [List practices.] In the next three months, will I? a. _________________________ 1 2 3 4 5 b. _________________________ 1 2 3 4 5 c. _________________________ 1 2 3 4 5 d. _________________________ 1 2 3 4 5 e. _________________________ 1 2 3 4 5

(38)

3. The expected outcomes for this training are listed in the left column. For each outcome please:

- Write a “B” on the scale to show your level of achievement when you Began the training today.

- Write an “N” on the same scale to indicate your level of achievement Now. 2. Circle the appropriate number to indicate your degree of understanding of the

listed topics BEFORE the training and AFTER the training.

Expected Outcomes Achievement Scale

a. Participant will increase knowledge about...

b. Participant will increase ability to…

c. Participant will gain confidence in…

d. _________________________________ e. _________________________________ 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 After training My understanding [List topics.] a. ___________________ 1 2 3 4 1 2 3 4 b. ___________________ 1 2 3 4 1 2 3 4 c. ___________________ 1 2 3 4 1 2 3 4 d. ___________________ 1 2 3 4 1 2 3 4 Before training Very Little Very Little

Some Quite Some

a Bit Quite a Bit A Lot A Lot

(39)

4. FUTURE PROGRAMMING

End-of-session questionnaires provide an opportunity to hear from participants about what their additional educational needs are, what preferences they have, how they best learn or would like to learn. This information is helpful in planning future programs or modifying existing efforts. Remember, though, that you are only receiving input from the people who came to the event, not all potential participants.

A. Needs Related to the Current Session

What do participants feel they still need to know about the topic at hand? In what areas do they feel they could use more practice? What is the best way to provide this additional programming? If you collect this information, it is important to respond to participant feedback. This shows that you take their opinions seriously.

1. Would you like more information on any topic we covered today? a. Yes

b. No

c. If yes, please list your questions or topics.

2. What would you like covered that wasn’t covered today?

3. What kind of follow-up on today’s workshop do you think would be helpful?

4. What assistance do you need in order to put into practice the ideas covered in this video conference?

5. Where do we go from here?

6. What additional information do you feel you need?

7. Do you have any suggestions that might help the group continue its learning?

8. Are there any topics you feel should be added to this session?

(40)

C. Other Planning Information

1. In preparation for our next session, we’d like to get some idea of what problems people are having related to [….]. Please describe briefly some issues or problems you’d like addressed.

2. In planning for our next workshop, it is helpful to know what people are

currently doing. Please check any items in the list below that you currently do. [List activities.] ____ a. ______________________________________ ____ b. ______________________________________ ____ c. ______________________________________ ____ d. ______________________________________ ____ e. ______________________________________ ____ f. ______________________________________

B. Topics for Future Sessions

1. What would you like to see addressed at a future field day? (Check all that interest you.) OR (Please check TWO that interest you the most.)

[List potential topics.]

____ a. _____________________________________ ____ b. _____________________________________ ____ c. _____________________________________ ____ d. _____________________________________

____ e. Other, please specify: _____________________________________

2. Please recommend topics and potential speakers for future training.

3. What topics would be of interest to you in future workshops?

(41)

3. Which of the following issues do you think are most important for us to address at an upcoming meeting? (Circle ONLY THREE.)

[List issues.] a. ________________________________ b. ________________________________ c. ________________________________ d. ________________________________ e. ________________________________ f . ________________________________

4. How do you think we might get more people involved in these meetings?

5. What method of delivery do you prefer? (Check all that apply.) a. ___ Small group discussion

b. ___ Lecture

c. ___ Large group discussion d. ___ Panel discussion

e. Other: ________________________________

6. How do you learn the best?

7. Which of the following techniques help you learn the best? (Check all that apply.) a. ___ Small group discussion

b. ___ Lecture c. ___ Role playing d. ___ Individual work

e. ___ Computer assisted learning f. ___ Video taped examples g. ___ Sharing by participants

h. ___ Other ____________________________

8. To what extent did the following influence your attending this conference: Strongly Influenced Did Not

Influenced Influence a. Content ____ ____ ___ b. Location ____ ____ ___ c. Length of program ____ ____ ___ d. Price ____ ____ ___ e. Speaker(s) ____ ____ ___ f. Sponsor ____ ____ ___ g. Other: _________________________________

(42)

9. Please rank order the following factors to show how they affected your decision to attend this conference. (Place a 1 next to the most important factor, a 2 next to the second most important factor . . . to 6=least important.)

a. ____ Subject b. ____ Speakers c. ____ Recommendation d. ____ Price e. ____ Location f. ____ Time of year

10. How far would you travel to attend another clinic like this one? a. ____ 0 - 20 miles

b. ____ 21 - 40 miles c. ____ 41 - 60 miles d. ____ > 60 miles

11. How would you recommend that we advertise this event?

12. How often should this event be held? a. ____ yearly

b. ____ every 2 years c. ____ every 4 years

d. ____ other ____________________________________

13. What time and day of the week are the most convenient meeting times for you? a. Time ________________

(43)

5. PARTICIPANT BACKGROUND

Participant background information helps you know who is attending your event. By asking about a few key characteristics you can better describe who you are reaching with your educational effort. And, it helps you know which people you are not reaching. Who did you want to participate? Participant background information helps you see if you reached the intended audience.

Also, you may wish to include a few items that will help explain participant responses to the other questions on the end-of-session questionnaire. What characteristics are likely to influ-ence people’s responses to your other questions of interest? For example, the size of a

participant’s business may influence how useful s/he finds the information. Or, the number of children a parent has may influence his/her reaction to the parenting workshop. Also, you may be interested to know if participants who are similar respond in a similar way to your educational effort. For example, do farmers with the same type of operation respond simi-larly? This information will help you explain your results and help in planning future programs. You also may be interested in knowing about the participant’s previous experience, knowl-edge or contact with Extension.

There are a variety of participant background questions that are useful, but only include those that you will use. Such questions are typically placed at the end of the end-of-session ques-tionnaire. You may wish to preface these questions to help participants understand their purpose. For example, “In order to know who is attending our Extension programs, please answer the following.” Or, “As a basis for understanding your earlier answers, we would like to ask you some questions about yourself…” Or, “In order to know you better, we would like to ask you the following.”

Background characteristics you might ask about include: ! Age

! Ethnicity

! Marital status

! Family size

! Occupation

! Size of farm/business operation

! Education

! Employment status/position

! Place of residence

! Level or proficiency in a skill

! Previous contact with Extension

! Prior knowledge of topic

(44)

A. Personal Information

1. What is your current position? _______________________________________

2. What is your age? a. ____ Under 18 years b. ____ 18 - 24 years c. ____ 25 - 34 years d. ____ 35 - 49 years e. ____ 50 - 74 years f. ____ 75 or older

3. What is the highest level of education that you have completed? (Circle one.) a. Less than high school

b. High school graduate

c. Technical school or some college d. College graduate or beyond

4. Are you employed outside the home? (Circle one.) a. Full time

b. Part time c. No

5. Where do you live? (Circle one.) a. Rural farm

b. Rural community (less than 2,000 population) c. Town of 2,000 - 24,999 population

d. City of 25,000 - 50,000 population e. City of more than 50,000 population

6. Please provide the following information: a. Number of years in farming: __________ b. Total acres farmed

Total owned: __________ Total rented: __________

(45)

7. Who lives in the household during the week? (Check one.) a. ____ One parent

b. ____ One parent and one or more other adults c. ____ Two parents (or stepparents)

d. ____ Two parents and one or more other adults

8. How many years have you been in the apple business? ______ years

9. What is the business structure of your farm? a. ____ Family owned

b. ____ Corporation owned c. ____ Partnership owned d. ____ Individually owned

10. What agency or organization do you represent? ______________________

B. Additional Information

Other background information of interest has to do with participants’ prior knowledge of the topic, where they receive their information and the participant’s prior involvement with Extension. 1. Have you attended meetings on water rights before?

a. ___ No b. ___ Yes

c. If Yes, Where? ___________________________ Who sponsored it? __________________

2. To what extent do you use the following information sources for parenting advice? (Circle one number for each information source.)

Not at All Some Quite a Bit A Lot a. Physician or nurse 1 2 3 4 b. Relatives and In-laws 1 2 3 4 c. Friends 1 2 3 4 d. Books, newspaper, magazines 1 2 3 4 e. Extension 1 2 3 4 f. Child care provider 1 2 3 4 g. County health nurse 1 2 3 4

(46)

3. During the past year how often have you taken part in programs provided by the following?

How helpful were they? (Check one.) Approximate

Number of Times Little Some Very a. Trade groups _______ _______ _______ _______ b. Professional associations _______ _______ _______ _______ c. Private companies _______ _______ _______ _______ d. UW-Extension _______ _______ _______ _______ e. Vocational or business school _______ _______ _______ _______

4. Which of the sources below do you use for information on parenting. (Check all that apply.) a. ____ Magazines b. ____ Newspapers c. ____ Books d. ____ Health professionals e. ____ Friends f. ____ Relatives g. ____ Salespeople h. ____ Teachers i. ____ Extension j. ____ Religious community

k. ____ Other, please list: ____________________

5. In the list below, which do you read on a regular basis. (Check all that apply.) a. ____ Agri-view

b. ____ Country Today c. ____ Grassland Farmer d. ____ Wisconsin Agriculturalist e. ____ Extension newsletter

(47)

6. How did you learn about this workshop? (Please check ALL that apply.) a. ____ Newspaper (name) __________________ b. ____ Magazine (name) __________________ c. ____ University announcement d. ____ Business associate e. ____ Friend/neighbor f. ____ Extension office g. ____ Other, please specify:

7. Where did you learn about this satellite program? a. ____ Brochure mailed to you directly

b. ____ Newspaper (Which one?) ________ c. ____ Brochure given to you

d. ____ Word of mouth e. ____ Internet

f. ____ Journal or magazine (Which one?) ________________

8. Was this your first Extension field day? (Circle one.)

a. Yes b. No

10. How many workshops conducted by a Cooperative Extension agent have you attended in [year]? _____________

(48)

BIBLIOGRAPHY

Dixon, N.M. 1990. “The Relationship Between Trainee Responses on Participant Reaction Forms and Posttest Scores.” Human Resource Development Quarterly, 1:2, 129-137.

Kirkpatrick, Donald L. 1998. Evaluating Training Programs: The Four Levels. 2nd Edition.

San Francisco: Berrett-Koehler Publishers, Inc.

Le Rouzic, V. and Cusick, M.C. 1998. Immediate Evaluation of Training Events at the Economic Development Institute of the World Bank: Measuring Reaction, Self-Efficacy and Learning in a Worldwide Context. Paper presented at annual American Evaluation Associa-tion Meeting, Chicago, November 1998.

Marshall, Mary. 1996. What Do You Want to Know? Collecting Information on Program Results. Texas Agricultural Extension Service. Texas A&M University, College Station, TX. Mimeo.

Phillips, Jack J. 1997. Handbook of Training Evaluation and Measurement Methods. 3rd Edition. Houston, TX: Gulf Publishing Co.

Rockwell, Kay and Kohn, Harriet. 1989. ‘Post-then-Pre Evaluation.” Journal of Extension, Summer 1989. URL: http://joe.org/joe/1989summer/a5.html

Steele, Sara. 1980. End-of-Meeting Information Sheets. University of Wisconsin-Extension, Madison, WI. Unpublished.

Steele, Sara. 1995. “Looking for More Than New Knowledge.” Journal of Extension, 33:3. URL: http://www.joe.org/joe/1995june/iw3.html

University of Wisconsin-Extension publications Planning a Program Evaluation, Observa-tional Techniques and Questionnaire Design: Asking Questions with a Purpose are available from UWEX Cooperative Extension Publications, 45 N. Charter St., Madison, WI 53715; Phone: (608) 262-3346; FAX: (608) 265-8052; or on the Internet at http://www.uwex.edu/ ces/pdande/Evaluation/evaluat.html

(49)

APPENDIX A

RETROSPECTIVE EVALUATION—

THE POST-THEN-PRE DESIGN

Appendix A is an article, “Post-then-Pre Evaluation: Measuring Behavior Change More Rapidly” by S. Kay Rockwell and Harriet Kohn (Journal of Extension, Summer 1989). It explains the logic behind retrospective evaluation and includes several detailed examples of end

Figure

Table 2. Sample practices included on self-report post-then-pre instrument.

References

Related documents

So there is peace of mind in knowing that Unionen’s income protection insurance is included in your membership fee and supplements your unemployment benefit and any severance pay

In particular, we find that the activity and other productivity indicators measured by the capital turn over ratio as measured by the sales/capital ratio vary across firms over

Using the fact that flux is equivalent to force and potential to velocity, the following is the mobility analog representation of the mechanical system given in

encoding the alpha subunit of a Saccharomyces cerevisiae G protein indicate a role in mating pheromone signaling. GPA1, a haploid-specific essential gene, encodes a yeast homolog

Neither our contract rights nor rights under a statutory lien apply to this account if: (a) it is an Individual Retirement Account or other tax-deferred retirement account, or (b)

The Chairman, Mr John Grogan, Mr Stephen Hepburn, Lady Hermon, Dr Alasdair McDonnell, Mr Denis Murphy, Stephen Pound, and David Simpson visited Northern Ireland in connection with the

For Calabi-Yau threefolds that do not admit any genus one or elliptic fibration, it would be interesting to understand whether there is some underlying structure to the

 The following general activities will have to be performed regarding Article 57 database: A full gap analysis to compare the current XEVPRM and the ISO IDMP format , data