Embedding evaluative learning in TZ training

Following our completion of the three initial phases of the TZ evaluation programme we ran a short workshop with the WMU staff and the TZ consultants on the work. One of the questions which emerged for the WMU was how to support teams and team development. This was a newly acknowledged gap,, as to date the TZ programme had put most effort in being effective at the task-focused elements of cleaner production training, i.e. the cyclic stages of assessing and analysing waste, implementing and monitoring changes, and in particular supporting teams to identify projects where they could make easy gains. The tension in the TZ programme between emphasising process and task is common to change- driving programmes in general. Many programmes oscillate from one perspective to the other, first emphasising ‘getting the job done’ and then swinging back to a process effort in order to develop more capacity for the long term (Kilvington & Allen 2001).

The WMU recognised that most of the technical consultants they employed to work with the teams had little or no experience in supporting social process elements of change. At the same time, the WMU were also aware the TZ teams’ evaluation process had been generally positively received by participating teams, two of whom commented that they regarded it as

useful part of the training they received through the TZ programme because it got people thinking. Following this feedback the WMU asked us if we were prepared to continue with the teams’ evaluation process as an integrated part of the TZ training. We proposed a process where we would pass on the checklist approach as a training tool to the TZ consultants and partner with them in their first round of evaluations. However, our assessment was that this was of limited success. As with the TZ teams themselves, the competency and confidence with this aspect of team work varied with the consultants. While some indicated they could happily see themselves integrating this into their ongoing work with companies, others felt the depth of expertise needed to adequately facilitate a truly reflective group evaluation was beyond them.

Subsequent iterations of the TZ programme experimented further with ways of supporting both teams and organisations to achieve long-term sustainable business practice. Most recent generations of the manufacturing company programme have seen it revert to a technical advisory approach to waste minimisation.

5.7

Summary – evaluation and social learning in the TZ programme

The TZ teams’ evaluation was designed to improve knowledge about how teams were functioning, and to uncover barriers and success factors for teams as agents of

organisational change, for use by the CCC to improve the long-term impact of the TZ company training programme. Most importantly it was oriented to set up a process that enabled teams to examine their own situation, and become more aware of their

achievements and limitations and empowered to affect their own productivity.

The TZ teams’ evaluation project illustrated a significant degree of success was possible in increasing the capacity of teams to both manage themselves, and their tasks, and enable them to investigate areas of the operation that were important to how effective they could be as instruments of change within an organisation. The use of a checklist of key aspects of group process, when coupled with a structured opportunity to assess these on a group-by- group basis, was a compact way to introduce teams to substantive learning about group processes. It ensured the reflection process was grounded in real experience and

A substantive criticism of resource use efficiency and cleaner production programmes is their failure to appreciate organisations as social systems and consequently their tendency to take a mechanistic approach to supporting innovation (e.g. concentrating on structural elements of teams, or relying on simplistic recipes for behaviour change). The TZ programme already encouraged thinking about organisations as technical systems; the teams’ checklist evaluation expanded this to thinking about the organisation as a social system. In particular it encouraged practical self analysis of the groups themselves, their behavioural norms and practices and their relationship to the organisation. The teams’ evaluation approach therefore illustrates a way to link both technical and process

management and support an overall questioning approach to problem solving that uncovers hitherto unthought-of non-technical influences on the change process.

Facilitation of reflection was critical to the effectiveness of the checklist review approach. This reflection process rested on both the skills of the facilitator and the position of the evaluation as an integral event in the training programme, i.e. coupling it with the same processes of analysis and review that were the mainstay of the ‘measure to manage’ approach to the TZ programme. A limitation of the self-reflection-based approach is the difficulty of overcoming existing resistance to reflecting on performance. The propensity for learning and development was different in each of the teams and often a manifestation of the capacity for innovation inherent in the organisation itself. The impact of such interventions as the TZ team’s evaluation can be magnified or minimised by the overall context in which they sit.

Another limitation of the TZ teams’ evaluation approach is its dependence on expertise within the programme to carry out the role of facilitating the enquiry. In the teams’ evaluation project we were unable to pass on the checklist-based teams evaluation to the existing consultants employed in the programme, despite the success of the approach in supporting team development, because they were foremost recruited for their technical expertise in waste minimisation, and either lacked skills or interest in extending their role to incorporate social process training.

The teams’ evaluation checklist was made freely available via the CLEM website. Over the following years a steady stream of interest in the approach has been evident from extremely diverse audiences, ranging from large commercial companies (e.g. Hewlett Packard)

most recent enquiry (March 2010) from the American Academy of Family Physicians, whose intention was to use the material in training to enhance the practices of family physicians.

Overall the TZ teams’ evaluation performed several roles, which match the four potential areas in which evaluation can support social learning outlined in Chapter 3 (see Figure 3.3). In the first instance it was able to support the company groups and their development as effective vehicles for change in the organisation. This occurred in two ways: firstly the checklist self-reflection approach offered the teams new techniques to enquire and problem solve; secondly, the teams’ evaluation checklist, through its grounding in the social aspects of organisational learning and change, is an example of how evaluation can link participants to conceptual understanding that helps them to diagnose the social learning challenges they are likely to encounter and develop strategies to respond to these. In other words the evaluation approach enabled the teams to scope the problem situation.

Thirdly, the teams’ evaluation projected collated information about how teams were working in general. These were discussed in meetings with the WMU and were thus instrumental in supporting the WMU learning about how the programme was working (programme management). Finally by linking ideas about how groups work with organisational change the teams’ evaluation extended the knowledge about how training initiatives like the TZ company training programme can meet their wider goals of increasing the uptake of cleaner production and adoption of resource use efficiency approaches across organisations (research and development).

Three particular points of interest about the potential of evaluation to support social learning emerge from the TZ programme. The first of these is the importance of being able to

negotiate a learning-based role for evaluation. This in itself was reliant on the existing responsive and adaptive capacity of the WMU. Similarly, and secondly, the TZ teams’ evaluation was able to utilise an already existing orientation towards developing a new learning capacity in participants, and was therefore not philosophically at odds with the programme in which it was operating. Thirdly, the project, as in the WCMP case story, illustrates that limited skills in consultants and programme managers can be a real constraint in embedding reflective, learning-based approaches in environmental management

Chapter 6

Frameworks for seeing across complex social systems