LEVEL 1—PARTICIPANT REACTION

Ho w participants like the cou rse and how they perceive its value

• Daily participant feedback (oral or written)

• Daily trainers’ meetings • Session/trainer evaluations • End -of-cou rse surv eys • End-of-course informal

discussions

Samples 9-1, 9-2 Samples 9-3, 9-4

Level 2: Participant Learning Understanding of principles and facts; demonstration of skills and techniq ues an d the a bility to apply them

• Kn owledge-based asse ssmen ts (e.g., pre- and mid course questionnaires)

• Co mpe tency-b ased skill assessm ents (e.g ., check list)

Samples 3-2, 7-1, 7-3 Samples 3-1, 6-1, 6-2, 9-5 POST-TRAINING:

Level 3: On-the-Job Performance Application of acquired

knowledge and skills as measured through change in on-the-job performance

• Site visit evaluations • Intervie ws with ind ividua ls

who comp leted training • Interviews with supervisors

Samples 9-6, 9-7

Level 4: Effect of Training

Cha nge in q uantity and /or qu ality of reproductive health services

• Service delivery statistics • Client exit interviews • Client record reviews • Ob servations at the facility

Adapted from: Kirkpatrick 1994.

LEVEL 1—PARTICIPANT REACTION

Reaction evaluation, conducted during and at the end of a course, provides a measure of participant satisfaction. Reaction evaluation answers the question: How satisfied are/were the participants with the course? It reveals feelings and perceptions from the individual’s point of view. The instructional designer must ensure that participant reaction techniques and approaches are built into the course design.

Information gleaned from reaction evaluation is extremely important and can be used to:

! Determine whether or not training is considered worthwhile. If considered worthwhile, training is more likely to be applied. On the

other hand, if participants are not satisfied with their learning experience, they may be less likely to apply what they have learned and may even advise others not to attend similar courses.

! Identify potentially serious problems in training design. For example, if a high percentage of participants complain that a particular topic is unclear, the topic content and presentation method should be reassessed.

In addition, reaction evaluation:

! Provides the participant with an opportunity to express her/his reactions to the course organizers (i.e., to comment on administrative and technical aspects of the training).

Reaction evaluations may take the form of:

! Daily participant feedback sessions (oral or written) ! Daily trainer meetings

! Session/trainer evaluations

! End-of-course surveys (written questionnaires) ! End-of-course informal discussions

Daily Participant Feedback Sessions

It is beneficial for trainers to evaluate the training process continually throughout the course. When conducted as a participant-led exercise at the end of each training day, daily reactions to training encourage participants to think and talk about what was learned during the day and to make suggestions for improvements to the entire group. One useful technique is to:

! Have participants (individually or in small groups) write on a piece of paper the two or three most important ideas or concepts they learned during the day, as well as suggestions for course improvement. ! Then, have each participant share with the group one or two items

from those they identified.

Daily Trainer Meetings

If two or more trainers are conducting the course, it is important that they meet briefly each day to discuss the participants’ evaluations of the day’s training activities, as well as each trainer’s personal assessment of the training. By doing this, trainers may identify elements of the training that need to be modified.

Session/Trainer Evaluations

Throughout the training, participants should be given the opportunity to evaluate the content for its overall relevance to their work and evaluate the trainers for their training ability. At the beginning of each course day, participants may be given copies of a session (Sample 9-1) or trainer (Sample 9-2) evaluation form. Participants should be encouraged to complete these forms after each training session and return them to the course organizer. The results of session and trainer evaluations can provide a basis for determining whether sessions need to be modified or whether a trainer’s clinical or training skills need improvement.

End-of-Course Surveys

Such surveys allow trainers to identify:

! The extent to which the course met participants’ expectations ! Aspects of the course that were the most or least helpful ! The relevance of the course content to the participants’ work ! The appropriateness of the training methods

! The extent to which administrative aspects of the course were satisfactory (e.g., the training environment, accommodations, travel arrangements, etc.)

Three important suggestions regarding the design and administration of these survey questionnaires are:

! Include close-ended questions so that trainers can easily tabulate the data and identify response patterns.

! Use a rating scale for questionnaire items. If the majority of participants rate an item very high or very low, this indicates that the reaction is shared, and suggests that the trainers should pay particular attention to addressing this item.

! Schedule sufficient time at the end of the course for participants to complete the questionnaire. Questionnaires should not be distributed late on the last day of training when participants are tired and may be preparing to depart.

Based on the survey results, a number of decisions can be made regarding course content, learning objectives, trainer effectiveness, course length and administrative arrangements. Sample 9-3 can be used to evaluate participant reaction to reproductive health training courses. Sample 9-4 is designed to be used by participants at the end of an IUD course.