Models of Training-Programme Evaluation

In the following sections, several training evaluation methodologies and models are presented.

2.4.8.2.1 Kirkpatrick’s Model

Kirkpatrick (2006) proposed a four-level methodology for evaluating the success of training programmes that comprises measurement of reactions to training, learning achievement, transfer and succeeding behaviour, and business outcomes. These measurement categories are shown in table 2.4.

Table 2.4: Kirkpatrick’s model of training evaluation

Level Description

Reaction Assessing what the trainees thought of the particular programme.

Learning Measuring the learning of principles, facts, skills and attitudes which were specified as training objectives.

54

Behaviour Measuring aspects of job performance which are related to the training objectives

Results Relating the results of the training programme to organisational objectives and other criteria of effectiveness.

Source: Kirkpatrick (2006, p.21)

The first level in Kirkpatrick’s model is the reaction or feelings that trainees have toward the actual training programme. While this outcome is an important starting point for evaluating programme outcomes, it is perhaps the least explored in existing studies identified for this meta-analysis. The second category in Kirkpatrick’s model is learning, concerned with knowledge outcomes, or ideas, information, and approaches from the training programme that are understood and retained by trainees. The third level is behaviour which is concerned with the actual on-the-job application of learned ideas, information, and approaches from the training programme. The final level, results, is broadly considered as the overall end results achieved (Alliger and Janak, 1989).

Kirkpatrick’s approach is the most common one utilised in the training field (Dubinsky et al., 2001). Many authors, such as Salas and Bowers (2001), Arthur et al. (2003) and Bramley and Kitson (1994) considered it to be the best known model for evaluation of training. Nevertheless, it has been criticised for the assumption that the levels are each related to the former and next level and that the model is too simple, failing to take account of the numerous intervening factors influencing training (Tamkin et al., 2002). Many objectives might be achieved through the evaluation of training, which in turn justify it.

Hashim (2001) stated that training evaluation may be carried out for many reasons, including gathering information that helps decision makers to improve training processes and facilitating participants’ job performance.

To conclude, Kirkpatrick and Kirkpatrick (2008) state that “trainees’ reactions affect learning. The term “trainees’ reactions” refers to the extent to which trainees are satisfied with their overall training participation, in other words the degree to which trainees are satisfied with all the elements of the training programme (i.e. trainer

55

performance, training environment and training components’ goals, content, material and process)”.

2.4.8.2.2 The CIRO Model

The CIRO model is another evaluation model broadly used in present organisations (Cooper, 1994). The four main components are context, inputs, reactions and outcomes. This model stresses evaluating the efficiency of managerial training courses, and as such it does not specify how measurement of training programmes for manufacturing employees can be done. Furthermore, it concentrates on measurements both before and after the training has been conducted. The core strength of the model is that the objectives (context) and the training equipment (inputs) are taken into account. On the other hand, this model does not assess behavioural change, which is a serious topic. In addition, the model recommends the implementation of measures through the training that can provide the training provider with essential information concerning the present state of the training programme, leading to developments.

Stuffelbeam (2000) recommended that the evaluation should contain of context, input, process, and output evaluation, and argues that these evaluations are completed at different steps throughout the programme progress. Tenant et al. (2002) affirmed that, to be active, training must have definite purposes and results, which apparently lead to organisations’ benefits.

A comparison of the two models is shown in Table 2.5.

Table 2.5: Comparison between the Kirkpatrick and CIRO evaluation models.

Criteria Kirkpatrick CIRO

Focused areas Reaction

Learning Behaviour Result Context Input Reaction Outcome

56

Strengths Identifies behavioural

change with an emphasis on the change of ability and application to jobs

Measures pre- and post- training to establish whether the objectives of the training were achieved

Weaknesses No focus on training

objectives, only post- training measures

Requires more training resources and has no behavioural focus

Source: Tenant et al. (2002) 2.4.8.2.3 A Four-Phase System

Lingham et al. (2006) suggested a four-phase system for training evaluation that focuses on designing the training programme, evaluating the preliminary programme, feedback and design of the evaluation tool, and continuing training and evaluation. They concluded that good evaluation schemes should be co-created and contain quantitative methods that capture features of content, applicability and trainee’ perceived significance of joining the training events (Figure 2.1).

57 Source: Lingham et al. (2006, p.337).

Much research has been focused on the success of training, and the focus has recently shifted from training output to process models asking why certain training has an effect and how it can be optimised (Kauffeld and Willenbrock, 2010). According to Mulder (2001), training organisations seek to acquire feedback regarding the quality of training programmes they have offered, as perceived by their clients, in order to make policy modifications if the results of the evaluation were disappointing. Mulder (2001) has developed an evaluation instrument consisting of a closed questionnaire which includes three groups of questions, administered before, during and after the training. For training organisations, training evaluation leads to improved training activities and to the building of a high reputation (Bimpitsos and Petridou, 2012).

McCoy and Hargie (2001) stated that no one model of evaluation is comprehensive and appropriate for all circumstances and, to evaluate efficiently, there is requirement for improved understanding of the nature of evaluation, its objective, and participants’ and employers’ needs. Tannenbaum and Woods (1992) identified many factors that can impact the training evaluation process: importance of training, change potential, scale,

58

nature and purpose, organisational culture, expertise, cost and timeframe. Failure to address any of these factors could restrain the success of the evaluation.

The following training-related concepts have been identified (Brinkerhoff, 1989; Bratton and Gold, 2003; Holgado-Tello et al., 2006; Kirkpatrick and Kirkpatrick, 2008):

• Reactions: trainees’ reactions regarding training goals, content, material, trainers, environment and training process;

• Learning: trainees’ acquired knowledge, skills, abilities and attitudes resulting from participation in training programmes; and

• Usefulness: trainees’ perceived usefulness, regarding their job, as a result of the training programme attended.

Kirkpatrick (2006), Yiu and Saner (2005) and Linghame et al. (2006) all recommended the use of pre-test and post-test in the evaluation activities of training programme participants.

However, Atiyyah (1991) argued that the evaluation approaches used in Arab training organisations are highly personal and their outcomes have inadequate utility in developing on-going programmes or designing new ones. Al-Athari and Zairi (2002) found that the use of a questionnaire, and observation, were the most popular training evaluation techniques used by Kuwaiti organisations, while ASTD (1997) reported that more than 90% of 300 US organisations used a questionnaire to evaluate training programmes. Atiyyah (1991) stated that the most widely used timing of the questionnaire is immediately after the programme is completed.

Hashim (2001) argued that most training programmes are evaluated only at the reaction level, associated with the terms “happiness sheet” or “smile sheet”, because reaction feedback is generally gained through a participating questionnaire managed at the end of a training event.