LEADERSHIP TRAINING AS AN EDUCATIONAL SYSTEM
5.5 Evaluating the effectiveness of a leadership training program
The starting point for the assessment of the effectiveness of leadership training is the basic assumption that leadership is the most important way of developing organizations and their cultures. Thus leadership training is the most important tool of controlled change.
With leadership it is possible to have an effect on everything that takes place in peace time Defence Forces. Therefore, when measuring for example the quality of training, leadership and the effectiveness of leadership training are indirectly measured at the same time, too (Vuorio, 2001 ). The following paragraphs present key tools for discovering the effectiveness of leadership in the hierarchy of objectives.
One part of systematic educational planning is evaluation that is related to execution. This makes it possible for the system to be dynamic and self-correcting. Evaluation means clarifying and measuring the results of training (Åhlberg 1992, 1 - 6). When the practical execution of training is analyzed, real self-correctiveness starts from the attitude of each instructor and educator to develop, not from ordered processes.
Several parallel concepts are related to evaluation, the most central of which are effectiveness, economy and productivity. Evaluation of effectiveness, i.e. the analysis of the actual effects of the given training, should take place at ali levels that are essential
training-wise. (Lehtinen, 1996.) The evaluation of the leadership training of the FDF should be based on the goal-setting of leadership training. ln a systematic approach to the leadership training evaluation, the areas being evaluated are:
1. The basic structure of the leadership training program.
2. lnternal evaluation of leadership training at all levels.
3. Persona! development of military leaders.
4. Leadership culture at local level.
5. Organization culture in the entire FDF.
6. The impact of military leadership training in the society.
5.5.1 The evaluation of the basic structure of leadership training
The evaluation of the basic structure of leadership training is a central part of the reform process of leadership training, and the results of the evaluation have already had an impact on both contents and method information of the curricula and teaching materia! of the program. Evaluation is carried out mainly on the basis of feedback information. This information has been collected from the leadership training of both conscripts and personnel.
The personnel, researchers and teachers of the Defence Staff, various units and the National Defence College (NDC) have participated in the collection and analysis of the feedback information. The nature of the evaluation of the program's paradigm-oriented basic assumptions is a long-term research activity. The evaluation of the program's method information is connected to the development of the training system of the entire FDF and it will be carried out within this framework.
5.5.2 lnternal evaluation of the leadership training system
Evaluation inside the leadership training system is continuous self-evaluation development at college level. Separating from the results the exact part that is the result of training may be a problem (Piirainen, 1996). At college level, results of effectiveness can be classified in the following way:
• experiences of know-how,
• willingness to develop oneself (motivation result),
• goal-bound performances in learning situations (achievement of goals),
• development of intellectual, social, etc. skills,
• fulfilling the needs of students. (Kilpinen et al, 1995.)
The feedback system of the NDC is one example of efficiency evaluation of leadership training. lt has been evaluated, that the process of total quality in evaluating the learning results is in a very high level in the NDC (Laine et al 2001, 12). The feedback system consists of the following parts:
1) Course feedback: carried out by the teacher in charge at the end of the course, includes a feedback discussion with students that is based on analyzed numerical feedback; a development memorandum is drawn up.
2)Annua/ course feedback: every year, the degree divisions collect, analyze and report on students' views on the general arrangements of training and the quality of teaching/guidance.
3) Work environment surveys: the headquarters of the College collects, analyzes and reports on the views of hired personnel on the requirements, resources and cooperation based on which the main task, i.e. training, is being carried out.
4) Extemal feedback: approximately one year after graduation, the degree divisions collect feedback from officers and their superiors in service. The aim of the survey is to find out how well training and especially its short-term aims have responded to the challenges of the duties of service.
This feedback system can be supported with surveys that are department- or subject-specific. For the part of the development of leadership behavior, data in the form of self-evaluation has been collected from cadet courses just before graduation on the attitude towards continuous self-development. When it comes to major questions, the information produced by the feedback system should form the basis of decision-making for the College principal.
5.5.3 The evaluation of military leaders' individual development
Concerning the evaluation of military leaders' individual development, the principle at all levels is that the leader himself is responsible for the collection, analysis and filing of feedback information on his leadership behavior. During the development discussions that are carried out on this basis it is possible to tackle the actual factors of change between a leader and his superior. The leadership portfolio functions as the memory and instrument
of the development of leadership behavior. When the highest leadership of the Finnish Defence Forces is concerned, the evaluation is supplemented with competency assessment, in which information relating to leadership behavior is supplemented by assessing the competencies needed in expert tasks as well as in the roles of operative and strategic level leadership.
5.5.4 Evaluating the leadership culture at local level
Leadership culture at local level is evaluated mostly at the level of annual work environment surveys. ln units that train conscripts, regular follow-up surveys that focus on the quality of training function are used as tools of evaluation. This system is further supplemented by conscript and reservist surveys that are carried out at regular intervals.
Work environment surveys are targeted at the entire hired personnel. ln its current form, a work environment survey yields practical information from individual level to be used in annual follow-up. Among other things, the information deals with:
•
the leadership behavior of superiors,•
the division of tasks in a unit,•
work satisf action,• work motivation,
•
strenuousness of work,•
work space and tools,•
the flow of information in the unit,• the clarity of objectives,
•
cooperation, and• the willingness of personnel to develop .
The results are also followed and reported at the national level.
Attempts are made to anchor the training results to the amount and quality of the war time troops that are trained. ln the final assessment, a 'client's point of view' is taken into account, meaning that the war time commander assigned from the regular personnel is included in the assessment process. The new leadership training sets developmental needs for this process, because the increased responsibility of training and leadership of conscript leaders must be taken into account when assessing the final product. This is particularly important when the aim is to evaluate the persona! productivity of an instructor based on the quality of war time troops. Reservist surveys are typically carried out during
refresher training, the aim being to collect information about attitudes towards national defence as well as the structure and execution of the refresher training in question.
5.5.5 Evaluating the organizational culture of the FDF
The organizational culture of the FDF can be assessed by combining and synthesizing the results of the feedback system targeted at leadership culture. The development of the new leadership training program began for contents and method information as a kind of pilat project in the teaching of cadets at the NDC in fall 1995. Feedback and experiences acquired from the teaching of cadets were exploited in the further development of contents and method information. The training of conscript leaders was renewed using the intake 11/98 whose service started in July; the renewal process was led by the Training Division of the Defence Staff.
On these grounds, it is possible to estimate that at company level, the timing of the first milestones of the real evaluation will probably be around year 2006. At that point, the majority of company commanders and all the young instructors will have received new leadership training and the basic structures of the program, like feedback systems, have become established. Effectiveness can then be assessed with the previously presented meters and results can be reported like follow-up studies. lt will be possible to begin to answer the question: /s leadership cu/ture changing? ln this case the basic assumption is that if leadership culture is noticed to be changing towards deep leadership, it will be shown as development and productivity of activities as well as enhanced motivation and satisfaction among all personnel groups.
Organizational culture has to do with long term follow-up and research. Because leadership culture is a central part of organizational culture, tentative conclusions can be made for a ten-year timespan. ln the effectiveness of leadership training, the decisive phase is when all personnel in service has received leadership training that is based on deep leadership. This will naturally take several decades to happen.
5.5.6 Evaluating the effects of the leadership training of the FDF in the Finnish society
Evaluating the effects of the leadership training of the FDF in the Finnish society is a new area of research. Organizing and developing research in this area requires increasing
cooperation in research between the defence forces and the surrounding society, as have already been done in the book "Johtajakoulutus murroksessa" (Nissinen & Seppälä, 2000). To the researchers of the FDF, the new tools of leadership behavior evaluation offer opportunities for various follow-up and cohort studies. There are also no unsurpassable obstacles in sight for research that focuses on the follow-up and success of persons in their civilian life who had received leadership training during their conscript service.