The Best Practice Transfer Workshops are assessed against the general criteria outlined in the terms of reference for this study and developed in Annex 1. This provides a basis for the abstract evaluation of the workshops and the feedback from the stakeholders.
Relevance
First assessment of the Environmental Compliance Assistance Programme for SMEs (ECAP)
31
The workshop programmes were in line with the general ECAP objectives in that they raised awareness of the overall ECAP and its sub-objectives. Specifically, the workshops and the final conference conveyed relevant targets on the following:
Provided information on best practice cases to improve a company‟s/SME‟s environmental impact, especially on environmental management systems (EMSs)
Build up a network of environmental experts in business support organisations.
Discussed and present funding possibilities for ECAP-relevant projects.
The Best Practice Workshops focused on an exchange of experience between different member states and establish partnerships between different ECAP stakeholders, especially support organisations. With this the targets and content of the workshops and the final conference were in line with the ECAP objectives.
Coherence
The content of the workshops included information that was partly gathered by interested people and from the ECAP homepage, i.e. information on better regulation and environmental management systems.
Economy
Under the overall term “economy”, the dimension of the action, the timing and the budget spent is discussed.
Size and format of seminars: the demand of stakeholders for the seminars was satisfied and no interested person or institution needed to be excluded from the seminar. The workshop concepts foresaw a number of participants between 10 and 20.
In practice, a lower number of participants were counted (from 5 to 10 participants not including speakers and organisers). For this kind of pilot workshop the number of participants was acceptable.
However with such few participants, networking opportunities are limited. The participants missed in some cases the relation to their specific regional context.
The final conference reached the expected number of participants. The feedback of the participants was in general very positive. Many participants suggested that the events could be much longer to ensure more networking opportunities and exchange of information.
Timing: The Best Practice workshops are in line with the other EU ECAP workshops. The general workshops for capacity building were in 2007 and 2008. These workshops followed in 2009, with a more specific focus.
Budgeting: The budget for the workshops was invested in an efficient way. There is no obvious possibility to reach the objectives of the workshops/conference (information, awareness raising, networking) in a way that would be decisively cheaper.
The economy-related criteria were fulfilled. The workshops did not reach a sufficient number of participants to reach an effective networking with a wide spread over different member states.
Effectiveness
The effectiveness of the seminars was measured by two indicators:
The number of addressees having taken advantage of the seminars; and
The opinions of the participants as to whether the seminars were perceived as useful.
Regarding numbers of participants, the three workshops comprised between 5 and 10 participants plus speakers and organisers. The workshop concepts had between 10 and 20 participants and demonstrated good discussions and networking opportunities. The final conference was attended by 70 participants.
The questionnaires from the workshops and the conference and the stakeholder survey show that participants largely responded positively to the events. The workshops helped the participants to improve their service for SMEs. A limited number of participants gave the feedback that the
32
transferability of the best practice examples was limited, due to the low number of participants at the workshops the limited networking opportunities.
The participants at the final conference gave a very positive feedback on networking possibility, the basis for establishing an ECAP network, origin of participants and speaker‟s contribution. They got a good overview of ECAP‟s services and that the examples are transferable to a large number of Member States. They also suggested that the duration of the event could be extended to have more possibilities for exchange of information and experience between the participants.
The effectiveness of the seminars could certainly have been even greater by having more participants.
The issue of transferability could be dealt with by presenting more examples and specific information in specific regions. In addition, participants should be encouraged to think carefully about whether there are elements of the examples provided that could be applied to their country.
Efficiency
The delivery of general knowledge of ECAP could be more effectively disseminated using the ECAP homepage. The workshops were good, when they showed very specific information, examples, etc.
The workshops were also helpful for discussing the problems faced by SMEs.
Sustainability
The workshops further developed the capacity building seminars commissioned by the EC.
“Sustainability” is promoted by the EC ensuring that further workshops and trainings opportunities (on-the-job trainings/study tours 2010) are carried out. No other national action has been reported.
Stakeholders responding to the survey said they took advantage of the opportunities to network at the conference and the workshops. The events enabled networking between stakeholders of different countries, allowing ECAP experiences, problems and successes to be shared. The seminar has intensified networking and knowledge gathering/exchange between stakeholders of different member states. Follow up projects were mentioned by a limited number of stakeholders.
The sustainability of the seminar series has therefore been realised through raising long-term awareness of ECAP, interlinking stakeholders for exchanging problems, sharing experiences or solutions and a limited number of follow up projects.
Utility
Most participants viewed the seminars as useful for overcoming problems. However, some individuals complained that the transferability of the best practice examples were limited for their regional context.
Furthermore the ability for networking is lowered with the smaller number of participants. Also the networking activities should be given more time in the schedule of the event.
As a result, the seminars have been useful for the majority of participants but the aspect of practicability and transferability of best practice cases should always have a high priority in the seminars. Also there should be scheduled time for networking activities during the event.
Consistency
Participants found that the workshops were integrated well with other activities and workshops.
Allocation / distributional effects
The distributional effects of the seminar series is assessed with regard to two different aspects:
a.) Was additional effort focused on those SMEs (sector wise) with the largest environmental impact?
The workshops and conference were designed to transfer best practice examples between Member States and SMEs and to develop an informal ECAP network. The events did not focus on sectors with especially large environmental impact. The workshops should address support organisations in general, not sector-specific organisations. The focus was more the exchange between different countries.
First assessment of the Environmental Compliance Assistance Programme for SMEs (ECAP)
33
b.) Was effort focussed on geographic location of SMEs, considering that those in less well developed EU states may have a worse track record regards environmental pollution?
Every workshop took care about a special topic and then was open for stakeholders from all member states. Because of the limited number of participants only organisations from some member states participated at the workshops.
Acceptability
The seminars were acceptable to the stakeholders. No extremely negatives opinions were notified to the interviewers.
4.4 Conclusions and recommendations
The best practice and networking workshops in 2009 were designed as initial pilot events which would contribute to the development of an ECAP network where the exchange and transfer good practice between stakeholders in different MS would take place. It would also enable stakeholders to form partnerships in order to widen their services to improve SMEs‟ environmental performance. This pilot project was designed to explore the effectiveness of organising workshops for SME support organisations, hence follow-up activities and evaluation were deemed of special importance.
Considering the levels of participation at the workshops and the feedback given, particularly the workshop on better regulation and on private and public networks, it seems that they did not arouse too much interest. Participant numbers ranged between five and 10 persons when not including the speakers and organisers. The workshop on simplified EMS attracted only a little more interest with 11 participants. Yet, it should be taken into consideration that organisers only foresaw 10-20 participants attending, in order to allow for interactivity between participants and for the presenters of the best practice cases to exchange their experiences.
The feedback from the questionnaires one year after the workshops was scarce (this is in contrast to the training workshops which are discussed in Chapter 3). It can be assumed that the themes of the pilot workshops were too abstract or too specific to attract significant interest from SMEs or SME support organisations and leave a lasting impression. Another reason for the lack of interest could be that the issues of simplified EMS or the tools of better regulation are already treated by specialised websites like ECAP‟s. Abundant information is available and special workshops may no longer be necessary or of interest.
By contrast, the final conference concluding the pilot scheme succeeded in attracting some 70 participants. The conference included a very varied programme and dealt with a multitude of issues. It could therefore be seen as a general information event aimed at a broader audience, with more possibilities for networking.
To rank the pilot workshops according to the assessment criteria given in section 2.2, the following assessment can be given:
Were the workshops practical enough?
The workshops were likely not practical enough in the sense that interested parties could learn new things i.e. things that could not easily be learnt by self-training, or of interest for them to be put into practice directly. The low participation rate is indicative of this. Yet, the representatives of the different best practice cases could exchange their experiences among themselves and with other participants.
Were the workshops considered an appropriate tool to build local environmental expertise in SME support organisations?
The low attendance rates also indicate that the workshops have failed in considerably helping to build local environmental expertise in SME support organisations that did not previously have such expertise. The concluding conference however was an opportunity to build environmental expertise.
Have any follow-up actions been organised (project element) and contacts maintained with other participants (network element)?
The conference in particular provided an opportunity for networking and has given some participants concrete ideas of projects. The workshops could only facilitate networking to a limited extent given the
34
low attendance. From the workshops as a whole, no follow-up actions in MS (e.g. regional/local level) have been reported.
The workshops have shown how to raise awareness of best practice in compliance assistance and networking possibilities. They can also be replicated at local/regional level. Thus, as a pilot project they have fulfilled their purpose despite the lack of evidence of uptake at national/regional/local level.
In order to have been as successful as the training discussed in Chapter 3 it is recommended that the workshops and conference should have:
Dealt with practical issues of interest to a broad audience of SMEs and SME support organisations;
Concentrated on issues/facts which cannot be easily be found or read online (on specific websites e.g. ECAP);
Included discussions with best practice practitioners and “on-site” demonstrations designed to convey lessons that can be put into practice directly by SMEs;
Allowed greater time for networking.