Ex-Post Evaluation of Projects and Activities Financed under the LIFE Programme

Ex-Post Evaluation of Projects

and Activities Financed under

the LIFE Programme

Final Report

Part 1: Methodology and description of the

LIFE Programme and its policy context

Julyl 2009

Directorate General Environment, Unit E.4.

LIFE

Ex-Post Evaluation of

Projects and Activities

Financed under the LIFE

Programme

Final Report

Part 1: Methodology and description of the

LIFE Programme and its policy context

July 2009

Prepared BIM, TVGU, STEH, ROKR Checked NEO, IL, TIH Approved BIM

This report has been prepared as a result of an independ-ent evaluation by COWI being contracted by the Director-ate General Environment

The views expressed are those of the Consultant and do not necessarily reflect those of the European Commission.

Table of Contents

Part 1:

Methodology and description of the LIFE

Programme and its policy context

2

1

Introduction 2

2

Methodology 3

2.1

Evaluation design

3

2.2

Approach to data collection and analysis

5

2.3

The evaluation process - activities, time and

resources 12

3

The LIFE Programme and its policy context

16

3.1

The anatomy of the LIFE Programme

16

3.2

Overview of LIFE projects during the period 1996 to

2006 20

Table of Appendices

Appendix 1

Indicators for assessment of results and impacts

Appendix 2

Detailed questions, success criteria and indicators

Appendix 3

E-survey questionnaire

Appendix 4

E-survey responses and data validity

Appendix 5

Nature and Environment projects selected for in-depth study

Appendix 6

Third country projects selected as case projects

Appendix 7

List of people met

Appendix 8

Bibliography

Part 1: Methodology and description of the LIFE Programme

and its policy context

1 Introduction

This document constitutes Part 1 of the final report in connection with the Ex-post Evaluation of

Projects and Activities under the LIFE Programme.

This first part of the report provides the background for the evaluation, explaining the methodology used

(Chapter 2) and some key characteristics of the LIFE Programme and the projects co-financed under it

(Chapter 3).

2 Methodology

This chapter presents the evaluation methodology applied. The methodology was developed during the

inception phase of the evaluation based on the Terms of Reference (TOR) and the Consultant's proposal

and following the EU Evaluation Guidelines

, and the standard evaluation criteria as specified in these

guidelines. A comprehensive report on the development of the methodology was given in the Inception

Report.

This chapter provides an overview of the evaluation design in section 2.1, the approach to data

collection and analysis in section 2.2 and, finally, in section 2.3, an overview of the evaluation process

is provided.

2.1 Evaluation

design

2.1.1 Evaluation

criteria

During the inception phase, the evaluation criteria suggested in the TOR were reviewed in light of the

objectives listed in the TOR and the requirements of the EU Evaluation Guidelines. This led to a

revision of the criteria thereby narrowing and focusing the scope of the evaluation. The following

criteria were identified as focus criteria: Effectiveness, efficiency, sustainability and utility. In addition

the criteria of acceptability and consistency were identified as secondary criteria.

2.1.2

Three components and nine evaluation questions

The evaluation is structured into three main components. Figure 2.1 illustrates how the evaluation

criteria apply.

Figure 2.1

Components and evaluation criteria

Source: Developed by Consultant

Results and impacts

The first component, comprising the bulk of the evaluation, concerns the evaluation of the results and

impacts generated by the LIFE Programme, i.e. the analysis of the actual LIFE intervention and its

effects, taking point of departure in the LIFE projects. The evaluation of results and impacts comprise

the three LIFE components: NAT, ENV and TCY. The analysis of LIFE NAT and ENV has been

carried out in the form of country studies

, which feed into thematic studies as illustrated by the left side

of the diagram in Figure 2.1.

Policy

The second component relates to the LIFE Programme as a policy and considers its relations to other

policies and how it is assessed by stakeholders.

Programme management

Finally, the component concerning programme management comprises an evaluation of organisation

and working methods of the LIFE Unit in managing the LIFE Programme, focusing on the effectiveness

and efficiency of the Unit in evaluating and selecting projects and monitoring selected projects.

The nine main evaluation questions which guide this evaluation emerge from the above mentioned

components.

1.

What are the results and impacts of the LIFE Programme?

2

Effectiveness:

2

. To what extent are the results of the LIFE Programme in line with the objectives?

3.

To what extent do the LIFE unit organisational setup, and the modalities for selection/monitoring of

projects, contribute to the effectiveness of the programme/projects?

Efficiency:

4

. To what extent is the cost at which results have been achieved reasonable when considering the

costs of comparative interventions?

5.

To what extent do the LIFE unit organisational setup, and the modalities for selection/monitoring of

projects, contribute to the efficiency of the programme/projects?

6.

Utility:

To what extent do the results and impacts generated by the LIFE Programme address the

environmental issues facing the EU and the stakeholders involved?

7.

Sustainability:

To what extent are the positive results and impacts generated by the LIFE

Programme likely to last after the termination of the projects?

8.

Consistency/coherence:

To what extent has the LIFE Programme been

complementary/supportive/contradictory to other EU policies and programmes?

9.

Acceptability:

To what extent do stakeholders accept the LIFE Programme and LIFE projects?

These nine evaluation questions are supported by a number of sub-questions and a detailed framework

for assessing each question in the form of success criteria, indicators and data sources. Appendix 2

provides an overview of these for each of the components in the evaluation. The framework described in

Appendix 2 is used as the reference guide for data collection and analysis as is further described in the

section below.

2.2

Approach to data collection and analysis

Data collection and analysis was performed in accordance with the framework provided in Appendix 2.

All data collected was used to validate (or negate) the success criteria, which fed into the analysis of the

evaluation questions and the formulation of conclusions and recommendations. The process varied for

the evaluation's three components and is therefore described in separate sections in the following

sections.

2.2.1

The results and impacts component

The evaluation of results and impacts implied two major steps, as illustrated in Figure 2.2. The first step

was an assessment of the effects (results and impacts). On the basis of the evaluation on effects, an

assessment on effectiveness, efficiency, utility and sustainability was made.

Figure 2.2

Evaluation of results and impacts

Source: Elaborated by Consultant

Division into two steps emphasises the fact that the assessment on the effects (results and impacts) of

the LIFE Programme is an output in itself and that an assessment is necessary in order to gauge

effectiveness, efficiency sustainability and utility.

Step 1: Assess effects

The assessment of effects requires the identification of indicators which can be used to describe the

effects. For the Nature and Environment components the indicators established are described separately

in Appendix 1. For the Third Countries component, the indicators have been integrated in the

framework described in Appendix 2. The Nature and Environment indicators are divided according to

the following main effect categories:

•

Environmental (improved state, reduced pressure, etc.)

•

Management (improved management, planning, administration)

•

Awareness (increased awareness)

•

Demonstration ("showcase effect")

•

Innovation (development of new technology or improvement of processes)

In the assessment of effects, we distinguished between results and impacts. We defined results as the

end-of-project situation, as reported in final reports and project summaries. For on-going projects,

results were assessed based on expected results as reported on in mid-term reports, etc. Impacts were

defined as the medium to long term effects in the period after finalisation of the project and assessed

primarily on the basis of e-survey responses and in-depth project studies. The completion dates of the

projects obviously vary considerably and some projects are still "on-going" (in progress). In general, the

assessment of impacts has greater validity for older projects. Impacts for recent or on-going projects

were assessed based on expected impacts, taking into account the results and relevant framework

conditions.

We used three main sources of project level data, as illustrated in Figure 2.3 below. In addition to

amassing the project level data, we have also carried out interviews with national focal points

(key/relevant country representatives), the monitoring team, DG Environment policy units and LIFE

Unit staff. The data collected through these interviews is thematic rather than project-specific and thus

serve as a fourth data/information source on results and impacts of the LIFE Programme.

Figure 2.3

Project level data used to assess effects of LIFE Environment and LIFE Nature

Source: Developed by the Consultant

The primary purpose of the

e-survey

was to gather data on the results and impacts of as many projects

as possible through targeting the project managers of LIFE projects. The questions in the e-survey were

organised according to the indicators for results and impacts. We had hoped for a response rate of 25%,

however, in total 275 projects managers of LIFE Nature or Environment projects completed the

questionnaire, corresponding to 15% of all Nature and Environment projects implemented during the

evaluation period. In addition, 219 project managers partly completed the questionnaire. We assessed

that 50 of the non-completed responses can be used as data sources, which brought the response rate to

18%.

Originally, the methodology was built on the assumption of a higher response rate. The implications in

using the results from the e-survey were different for the Nature and Environment components due to

the response rate, different characteristics of the portfolio of projects and the number of projects within

the themes. For the Nature component, with 771 projects in total and e-survey responses covering 158

projects the response rate was 20%. Furthermore, the Nature projects were dealt with as one large theme

and the projects within the theme were relatively homogeneous. For this reason the e-survey data was

assessed as representative of the total Nature project population and acceptable as a basis for forming

conclusions at aggregate (EU) level.

The e-survey was designed to provide input to all three components and consisted of two sections: One section aimed at project managers of LIFE Nature and Environment projects in the period 1996-2006 while a general section aimed at all stakeholders concerned with the LIFE Programme. The first section aimed to generate data on results and impacts of the projects as well as solicit the opinions and perceptions of the project managers of the management and administration of the LIFE Programme. The general section of the e-questionnaire aimed at obtaining comments and opinions on the LIFE Programme in general from a broad base of stakeholders. The e-survey questionnaire is included as Appendix 3.

LIFE Environment projects constitute a very heterogeneous group by comparison. Even when

considering the projects categorised under the same theme (e.g. air), there is much variance. At the same

time the response rate for Environment projects was also lower with 167 responses for 1076 projects in

total (16%). The response rate was lower for some Environment themes and higher for others.

Considering the variance in projects, the e-survey results were not considered as sufficient basis to form

conclusions. The thematic studies therefore reviewed a number of project summaries within each theme

in order to obtain an extra data source on project effects and ensure validity and authenticity. Appendix

4 provides more background of the analysis of the representativeness of the e-survey data.

The

105 projects selected for in-depth study

were selected according to a range of criteria in order to

attain the highest possible level of representation compared to the overall portfolio of Nature and

Environment projects. The criteria did not incorporate country or thematic specific criteria. In Appendix

5, the selection process, criteria and list of projects is described. The in-depth studies were performed

through review of final reports and monitoring files, interviews with project managers, and site visits

where relevant. A site visit has been made where this was logistically possible and allowed for actual

observation of results and/or impacts of the project. The in-depth project studies were performed as part

of the country studies but fed into the thematic studies as well. No projects were selected for the ENV

themes on forests and chemicals as there are very few projects, which fall under these two themes.

These thematic studies relied to some extent on desk reviews of selected projects instead of in-depth

project studies.

From the

BUTLER database

, we extracted descriptions of project results given in project summaries as

well as the project rating designated by the LIFE Unit in the more recent projects. In total 398 of the

1847 Environment and Nature projects have been scored.

The data collection and analysis related to LIFE Third Countries (TCY) was guided by specific

evaluation questions, success criteria and indicators, as described in Appendix 2. This is because the

LIFE TCY component was implemented with objectives that differ substantially from the LIFE Nature

and LIFE Environment components

. Consequently, the approach to data collection and analysis also

differs from that used for LIFE Environment and LIFE Nature. Data collection on LIFE TCY included a

desk review, interviews and project case studies. The desk review was based on the information

available on the LIFE Website, BUTLER (progress reports, monitoring reports and evaluation reports)

and information contained in the relevant reports/publications provided by the LIFE Unit. Interviews

were carried out with the key stakeholders in the LIFE Unit in DG Environment and in DG RELEX, and

DG AIDCO. In addition, 17 of the 179 projects co-financed by LIFE TCY in 16 countries, were selected

as case studies, including document review, telephone interviews with the national country coordinators,

the EU delegations and the beneficiaries. The selection procedure and the projects selected are

documented in Appendix 6.

During the evaluation period Cyprus, Estonia, Latvia, Lithuania, Malta, Slovenia and Poland were TCY countries and

became part of LIFE NAT and ENV upon their accession to the EU. However, the accession countries had the

possibility to apply for projects under the NAT and ENV components. Most accession countries made use of this option

and, consequently, only Cyprus and Malta have actually implemented TCY projects during the evaluation period. The

country studies in Malta and Cyprus will include TCY related questions and the data collected will feed into the TCY

study. Hence, the country reports (dealing with the implementation of LIFE Nature and Environment) on Malta and

Cyprus will not contain a specific assessment on the TCY projects. This assessment is made in the TCY report.

However, the country reports may make reference to TCY projects and their importance in relation to implementation

of Nature and Environment projects, if any.

Step 2: Assess evaluation criteria

The next step was to use the assessment of results and impacts to assess the LIFE Programme on the

criteria of effectiveness, efficiency, utility and sustainability. This involves comparison on the effects

with various "benchmarks", as illustrated in Figure 2.2 above:

•

Effectiveness compares results with

objectives

•

Efficiency compares results with costs against a benchmark of costs of similar interventions

•

Sustainability compares impacts with results and

also relates this to overall/global objectives

•

Utility compares effects with the

needs, problems and issues to be addressed

The terminology used in assessing effectiveness and efficiency in the country studies and thematic

studies is shown in Table 2.1.

Table 2.1

Terminology in assessing effectiveness and efficiency

Effectiveness - projects Effectiveness - policy Sustainability Very low None of the projects

effective

Targets generally not met No projects are sustainable

Low A few of the projects

effective

Targets met to a limited extent

A few projects are sustainable

Medium The majority of the projects effective

The majority of targets met The majority of the projects are sustainable

High All projects 100% effective with a few exceptions

Targets met with a few exceptions

All projects sustainable with a few exceptions

Very high All projects 100% effective All targets met All projects sustainable

This implies a need to specify which benchmarks to apply for this particular evaluation. This is reflected

in Appendix 2, which contains the list of evaluation questions together with associated success criteria,

indicators and sources of data. Some general observations with regard to the benchmarks highlighted in

the list above are highlighted below.

Defining the benchmarks relating to

objectives

was a challenge for this evaluation as it covered several

generations of LIFE with corresponding LIFE Regulations, which, in turn, contained similar but

different objectives. Based on the regulations (including the LIFE+ Regulation), we established one

"generic" overview of objectives as illustrated in Table 2.2.

Table 2.2

LIFE Objectives

Overall policy objective

To contribute to sustainable development;

To favour the integration of the environment into other EU policies;

To lead to new solutions for the environmental problems.

Intermediate policy objectives

To contribute to the implementation, development and enhancement of Community environmental policy and legislation

Specific objectives LIFE-NAT: To contribute to the implementation of Council Directive 79/409/EEC (Birds Directive) and Council Directive 92/43/EEC (Habitats Directive).

LIFE-ENV: To contribute to the development of innovative and integrated techniques and methods and to the further development of Community environment policy. Objectives for each theme are indicated in Appendix 15.

LIFE-TCY: To contribute to the establishment of capacities and administrative structures needed in the environmental sector and in the development of environmental policy and action programmes in third countries bordering the Mediterranean and the Baltic Seas6.

Source: Elaborated by the Consultant. Note: None of the LIFE regulations distinguish between overall policy objective

and intermediate policy objectives in the same as illustrated in the table. This distinction has been made by the

Consultant.

In relation to the benchmark for

costs of similar interventions

our intention was to identify standard

unit prices for the various types of results under each theme based on information available in e.g.

impact assessment for Directives, Directive specific financing plans, available sector specific studies,

etc. However, due to insufficient data this was generally not possible. The assessment on efficiency

therefore relied on expert assessments and the perceptions of key stakeholders.

The benchmark relating to

needs, problems and issues

was addressed with a point of departure in the

6th Environmental Action Plan and the associated documents, including thematic strategies. For third

countries, the point of departure was country documents, such as national environmental action plans.

2.2.2

The programme management component

The overall aim of the programme management component was to analyse the extent to which the

organisational set-up of the LIFE Unit and the modalities for selection and monitoring of LIFE projects

contribute to the effectiveness and efficiency of the programme. The programme management was

evaluated via a) our assessment and b) feedback from beneficiaries (e-survey and interviews) cf. below.

The combination of COWI expert assessment and beneficiary assessment allowed for a solid database

with which to evaluate programme management.

Further specification and elaboration of thematic objectives is done in connection with thematic studies and reflected

in thematic reports.

6

Other than the countries of Central and Eastern Europe which have concluded Association Agreements with the

Euro-pean Community.

Figure 2.4

Approach to programme management evaluation

Source: Developed by Consultant

COWI assessment of programme management

COWI's expert assessment of programme management was based on desk reviews, interviews and

comparisons to other programmes. On top of that, a LEAN inspired process optimisation analysis of key

processes monitoring processes, was undertaken. The analysis was conducted through two workshops

facilitated by COWI consultants, where LIFE Unit staff analysed work processes in detail and identified

potentials for efficiency gains. The aim of the LEAN analysis was to provide:

•

A measure of how man-power resources are spent in selection and monitoring processes

•

A description of 'the good flows' and 'the bad flows' in the programme management cycle

•

An increased focus on learning and improvement internally in the LIFE Unit

Beneficiary assessments

Through the e-survey and interviews with project managers and national focal points, the beneficiaries'

assessments of programme management were solicited. These data sources are combined and

triangulated in order to answer the evaluation questions related to programme management.

2.2.3 The

policy

component

This section deals with the policy oriented evaluation themes consistency (hereunder coherence) and

acceptability. These themes touch upon how the LIFE programme links and fits in with other policies

and instruments and how they are accepted by the broader public, both locally and nationally.

Coherence and consistency

When assessing consistency, the main issue is how the LIFE programme links to other EU programmes

and policies. The success criterion is for the programme to support and complement other instruments

and policies rather than to be funding exact same objectives (double funding) or on the other hand

contradictory objectives to other programmes and policies. We have checked consistency between the

relevant instruments, hereunder, most notably, the financial instruments under the Structural Funds,

Cohesion Funds and Framework Programmes. In relation to the LIFE Nature component, the

Community Agricultural Policy (CAP) and the Community Fishery Policy (CFP) are policies, which

potentially contradict with environmental policy in general and the LIFE Programme and this was also

analysed.

The data collection has partly been a desk study and partly an analysis of the interviews in relation with

the country studies. However, we have also relied on the programme management analysis of the project

selection and monitoring process. The assessment was based on the following:

•

Examining how the project selection process in a consistent way supports the EU policies

•

Observing project managers and LIFE unit staff perceptions of the level of consistency and

acceptability

•

Beneficiaries' assessment of programme policy consistency and acceptability

•

National focal points' perception of consistency and acceptability

Acceptability

The evaluation term 'acceptability' describes different target groups' attitudes towards the LIFE

Programme both in general and for specific LIFE projects. The main issue is to assess the level of

acceptability of the LIFE Programme among project managers via the e-survey. Acceptability can, for

obvious reasons, be difficult to asses in objective terms and to a large extent we have drawn on the

project managers' perceptions of local and national stakeholders' acceptance of the individual projects.

2.3

The evaluation process - activities, time and resources

Figure 2.5 shows an overview of the main activities in the evaluation. The evaluation was divided into

three overall phases: The inception phase, the data collection and analysis phase and the reporting phase.

Due to the complexity and size of the evaluation there was some overlap between the phases.

2.3.1 Inception

phase

The inception phase began on 24 July 2008, when the contract was signed, and ended on 6 September

2008 when the first version of the inception report was submitted to the Steering Committee. The main

activities during the inception phase encompassed:

•

Kick-off meeting with the LIFE Unit and informal meetings with LIFE Unit staff

•

Internal kick-off and inception workshops for the evaluation team

•

Collection of material on the LIFE Programme and ensuring access to ESAP and BUTLER

databases (on selection and monitoring of projects, respectively).

•

Preparation of consolidated evaluation methodology including formulation of evaluation questions,

success criteria and indicators for each of the LIFE components and themes to be evaluated

•

Set-up of a project database to store information retrieved from BUTLER to enable overview of

projects per country, theme, etc. as well as statistical analysis of projects and finance data

•

Preparation of inception report describing the methodology and approach

The Steering Committee provided comments to the inception report which was revised and extended to

cover the final version of the questionnaire to be used in the e-survey and reporting on activities in the

data collection phase. The final inception report was submitted 6 November 2008.

Figure 2.5

Main activities in the evaluation

Source: Developed by Consultant

2.3.2

Data collection and analysis phase

This phase of the evaluation formally began on 7 September 2008 and included activities, which

cross-cut all components of the evaluation and activities, specific to each component. These are described

below. This phase, which overlaps with the reporting phase, ended in mid-March when the country and

thematic studies were finalised.

Cross-cutting activities: Project database, e-survey, and selection of projects for in-depth study

The establishment of a project database containing information on all LIFE projects during the period

1996-2006 was initiated during the inception phase. The access to the BUTLER database granted to the

Consultant, unfortunately, did not allow the Consultant to make extractions from BUTLER. The LIFE

Unit was very helpful in supplying tables extracted from BUTLER and the database has been

continuously updated with new data throughout the evaluation process. A raw version of the database

created is submitted to DG Environment along with the final report.

The e-survey was launched 27 October 2008 and closed on 2 December 2008.

Selection of ENV and NAT projects for in-depth study: In the original time schedule for the evaluation

it was foreseen that the projects for in-depth study would be selected by 1 October. Due to data

collection difficulties there was considerable delay. On 19 December, the final list was sent to the LIFE

Unit (see details on selection process, criteria and final list in Appendix 5).

Programme management component

The activities under the programme management component were initiated immediately after the

inception phase and included the following:

•

Desk review of the relevant material (evaluation guide, standard administrative provisions/common

provisions, contract with monitoring teams, Vademecum, description of selection and monitoring

teams, and GHK-study on monitoring teams)

•

Interviews with staff from the LIFE Unit and monitoring team

•

Two LEAN workshops with LIFE Unit staff

•

Beneficiaries' assessment based on input from e-survey and country studies

•

Analysis and report drafting

Results and impacts component

This component included country studies (which encompass in-depth project studies), thematic studies

and the TCY study. The activities are described below.

Country studies

•

Country study guidelines: Guidelines were developed in a final draft version to be used in the

country study workshop on 1 October 2008. The guidelines comprise background on the

evaluation; description of outputs and tasks; templates for country study reports, project sheets

(reporting on in-depth project studies) and interview reports; and interview guides. The guidelines

have been developed continuously during the period and the latest version was issued 2 December

2008.

•

Country study workshop: On 1 October 2008, a country study workshop was held for the

consultants responsible for country studies. During this workshop, the templates and guidelines

were presented and explained and the approach to the country studies discussed.

•

Pilot country studies: In the time schedule developed in the inception phase, it was foreseen that

pilot country studies on Denmark and Sweden would be carried out and submitted by mid-October.

Due to the delays described above and more resources spent on e-survey, project selection, etc. than

originally foreseen, the pilot country studies were delayed. It was decided to focus on Denmark and

the pilot country report was submitted to the LIFE Unit on 17 November 2008.

•

Information package: Each country study consultant received an information package containing

data to be reviewed as part of the country study. In addition to this, the country study consultants

reviewed the latest national environmental action plan or similar document. The package included:

-

Reference to the LIFE web-site

-

Extractions from the database concerning numbers of projects and projects according to year,

theme, etc.

-

Project summaries for all projects in the country

-

CD with monitoring files

-

Project reports for projects selected for in-depth study

-

LIFE Publications concerning the country in question

-

Results from the e-survey relevant to the country in question

-

Names and contact details of relevant interview persons

•

The main country study activities in each study are listed below.

-

initial desk review of project summaries and suggestions for selection of projects for in-depth

study

-

a desk review of the above mentioned documents

-

interviews with national coordinators

-

interviews with LIFE Unit country desk officers and monitoring teams

-

in-depth project studies for selected projects, including interviews with project managers and

site visits to projects with observable results and impacts

-

compiling and assessing data on results and impacts according to indicator framework

-

assessing the effectiveness, utility and sustainability of the LIFE projects in respect to the

country's needs and issues

-

reporting, in the form of a country report (chapter in final evaluation report), and project

sheets, reports from interviews, etc. to be provided to thematic teams for further study.

Thematic studies

For both NAT and ENV themes, a categorisation of all projects (771 NAT and 1076 ENV) into themes

were made and documented to the LIFE Unit (see Part 4 for details).

The thematic studies comprised the following for each theme:

•

Studying relevant EU policy documents

•

Interviews with relevant staff in the LIFE Unit and DG ENV

•

Review of results of e-survey

•

Review of results of country studies, notably in-depth project studies

•

Review of project results as reported in BUTLER

•

Analysis/aggregation of project level data on effects to the thematic level

•

Assessing effectiveness, efficiency, utility, sustainability

•

Reporting

TCY Study

The main activities included desk study, 17 project studies, interviews with DG Environment officers

and EC Delegations, and analysis and report writing.

2.3.3 Reporting

phase

The reporting phase was initiated with the progress report submitted 6 January 2009, which was

structured in accordance with the structure for this draft final report. From January and until submission

of the draft final report 20 April 2009, we worked in a targeted manner within the report structure as

revised by comments received at the Steering Committee meeting concerning the progress report. The

report was subsequently revised again based on comments received resulting in the final report as

presented in this volume.

3

The LIFE Programme and its policy context

This chapter provides a brief overview of the main characteristics of the LIFE Programme and its policy

context. More detailed and targeted descriptions on various parts of the programme are provided where

relevant in sections of this evaluation report. The chapter is structured into two main sections: The first

section provides an overview of the main features of the Programme and the second section describes

the LIFE projects during the evaluation period.

3.1

The anatomy of the LIFE Programme

3.1.1

Generations of LIFE

The LIFE Programme has existed since 1992. Until 2006, it was implemented in four phases or

generations each governed by a separate Council Regulation. The current, fifth generation (LIFE+- not

encompassed by this evaluation), runs from 2007 to 2013. The evaluation period of 1996-2006 covers

three generations of LIFE:

•

LIFE II (1996-2000)

•

LIFE III (2001-2004)

•

LIFE III-extension (2005-2006)

3.1.2

Why LIFE? A description of the objectives

The scope and priorities of the LIFE Programme, as outlined in its governing regulations, have

developed over the three generations. However, the programme has maintained a dual objective of

implementation and development of Community environmental policy and legislation. Over the years,

there has been a trend towards increasing focus on implementation rather than development of policy

and legislation.

Table 3.1

Key developments of the LIFE programme

Phase Regulation Years General Objective of LIFE Key EU developments

LIFE I Council Regulation (EEC)1973/92

1992-1995 Contribution to the development and

implementation of Community environmental policy and legislation by financing:

(a) priority environmental actions in the Community;

(b) (i) technical assistance actions with third countries […] 1995: Austria, Finland and Sweden new MS LIFE II Council Regulation (EC) 1404/96

1996-1999 Contribution to the development and, if appropriate, implementation, of Community environment policy and legislation.

LIFE III Regulation (EC) 1655/2000

2000-2004 Contribution to the implementation, updating and development of Community environment policy and of environmental legislation, in particular as regards the integration of the environment into other policies, and to sustainable development in the Community 2001: Restructuring of DG ENV; and adoption of the EU SDS 2002: the 6th EAP adopted 2004: eastern enlargement with 10 new MS LIFE III extension Regulation (EC) 1682/2004 2005-2006 Unchanged 2006: renewed SDS adopted LIFE + Regulation (EC) 614/2007

2007-2013 Contribution to the implementation, updating and development of Community environmental policy and legislation, including the integration of the environment into other policies, thereby contributing to sustainable development. In particular, LIFE+ shall support the implementation of the 6th EAP, including the thematic strategies, and finance measures and projects with European added value in Member States. 2007: Romania and Bulgaria new MS Programme priorities clearly linked to the 6th EAP

Source: elaborated by consultant

3.1.3

Components in the LIFE Programme

The LIFE Programme was divided into three components during the evaluation period: LIFE

Environment, LIFE Nature and LIFE third countries (TCY).

LIFE Environment (ENV)

The scope of LIFE Environment has been fairly similar during the three generations of LIFE and has

focused on the development of innovative and integrated techniques, demonstration projects, and

development of Community environmental policy. The share of the LIFE Programme budget allocated

for LIFE Environment was 46% under LIFE II and 47 % under LIFE III and LIFE III extension.

LIFE Nature (NAT)

The specific objectives of LIFE NAT have remained fairly constant during the evaluation period and

have focused on the implementation of the Birds and Habitats Directives, as exemplified by art. 3.1 of

the LIFE III Regulation:

The specific objective of LIFE-nature shall be to contribute to the

implementation of Council Directive 79/409/EEC of 2 April 1979 on the conservation of wild birds,

Council Directive 92/43/EEC of 21 May 1992 on the conservation of natural habitats and of wild fauna

and flora and, in particular, the Natura 2000 European network established by the latter Directive".

Similarly to LIFE Environment, the LIFE Nature budget share was 46% and 47 % under LIFE II and III,

respectively.

LIFE Third countries (TCY)

The focus of LIFE Third Countries was on providing support to capacity building in the environmental

sector and on the development of environmental policy and action programmes in third countries

bordering on the Mediterranean and the Baltic Sea

. LIFE TCY was allocated 5% of the budget under

LIFE II and 6% under LIFE III and LIFE III extension.

Other

Under LIFE II, a separate budget was set aside for "accompanying measures needed to monitor,

evaluate or promote the actions undertaken during the first stage and under points 1 [environment,

hereunder nature] and 2 [TCY] and dissemination of information on the experience and the results

obtained of such actions." These actions were allocated 3% of the total budget under LIFE II.

3.1.4

The LIFE cycle

The LIFE Programme is basically organised in an annual cycle. Each year project proponents can apply

for LIFE co-financing according to criteria and guidelines announced by the LIFE Unit. The project

applications are submitted to the LIFE Unit by the Member States. The project applications are then

evaluated by the LIFE Unit and projects selected according to a set of criteria. During implementation,

the projects are monitored by the LIFE Unit. This cycle is illustrated in Figure 3.1 below.

Other than the countries of central and Eastern Europe which have concluded Association Agreements with the

Euro-pean Community and are referred to in Article 6(1). (LIFE III art 5)

Figure 3.1

The LIFE Cycle

Source: Developed by Consultant

3.1.5

The input: Budget

The LIFE regulations stipulate the total budget allocated to the LIFE programme for the phase to which

the regulation applies. The annual average budget of the programme has increased from phase to phase -

from EUR 112.5 million under LIFE II to EUR 158.6 million under LIFE III extension.

Table 3.2

The budget developments of the LIFE Programme

LIFE programme Years Budget

million EUR

Annual Average million EUR

LIFE I Council Regulation (EEC)1973/92 1992-1995 400 100

LIFE II Council Regulation (EC) 1404/96 1996-1999 450 112.5

LIFE III Regulation (EC) 1655/2000 2000-2004 640 128

LIFE III extension Regulation (EC) 1682/2004 2005-2006 317.2 158.6

LIFE + Regulation (EC) 614/2007 2007-2013 2,143.4 306.2

Source: the respective LIFE regulations

3.2

Overview of LIFE projects during the period 1996 to 2006

During the period 1996 to 2006, 2033 projects were approved for LIFE co-financing. These included

771 Nature projects, 1076 Environment projects and 179 Third Country projects. Of these projects, 291

are still on-going

. This section provides a brief overview of the projects. More specific overviews per

country and theme are provided in the relevant country and thematic reports. The data presented is from

the BUTLER database of the LIFE Unit. It should be noted that there are several uncertainties:

•

The data on commitments in the BUTLER database mostly stems from the original project budgets

and hence, does not reflect the actual payments made (or foreseen) as budgets are often changed

during the projects' lifetime. It was not possible to supply data on payments which could be aligned

with the BUTLER data as this data exists only in the accounting systems from where the necessary

tables could not be created.

•

The BUTLER database also includes the projects closed down. Through accounting data from

2004-2006, it was possible to identify some of these projects and delete them from the database, but

interview data indicates that there are additional projects in BUTLER, which were, in fact, never

fi-nanced.

Table 3.3

LIFE projects and commitments 1996-2006

LIFE Generation LIFE Environment LIFE Nature LIFE TCY Total

Projects Commitm ent (million EUR) Projects Average commitm ent (EUR) Projects Commitm ent (million EUR) Projects Commit ment(mi llion EUR) LIFE II 613 265.51 408 278.56 95 28.93 1116 573.00 LIFE III 325 215.58 247 219.37 53 20.16 625 455.12

LIFE III extension 138 134.78 116 139.28 31 12.84 285 286.90

Total 1076 615.88 771 637.21 179 61.93 2026 1315.01

Source: Data in BUTLER database

For the purpose of this evaluation on-going projects are defined as projects with end-year in 2009 or later as registered

in the BUTLER database.

The overall committed amount during this period amounted to EUR 1,315 million. As shown in Table

3.3, the number of projects approved annually decreased during the evaluation period whereas the LIFE

contribution to each project has increased.

Table 3.4

Annual average number of projects and commitment per project, 1996-2006

LIFE Generation LIFE Environment LIFE Nature LIFE TCY Total Projects per year Commitm ent per project (million EUR) Projects per year Commit ment per project (million EUR) Projects per year Commit ment (million EUR) Projects per year Commit ment per project (million EUR) LIFE II 123 0.4 82 0.7 19 0.3 223 0.5 LIFE III 108 0.7 82 0.9 18 0.4 208 0.7

LIFE III extension 69 1.0 58 1.2 16 0.4 143 1.0

Total 108 0.6 77 0.8 18 0.3 203 0.6

Source: Data in BUTLER database

3.2.1 Themes

For the purpose of this evaluation, we have divided Nature and Environment projects according to the

themes listed in Table 3.5. These themes have been inspired by the LIFE+ Regulation and their

categorisation is described in more detail in Part 4 on the thematic studies. Table 3.5 shows that the

Environment themes 'water', 'natural resources and waste' and 'strategic approaches' are the most

prevalent themes in terms of number of projects. Although less so this is also the case when looking at

the LIFE contribution. For the Nature projects, about three quarters are categorised as habitats project

serving to implement the Habitats Directive and one fourth as bird project and thus serving to

implement the Birds Directive. The number of "other" projects

is very small.

Table 3.5

Number of Environment and Nature projects and committed amounts according to themes

Natural resources and waste _{312 16.9% 183.7 14.7%}

Chemicals _{17 0.9% 9.5 0.8%} Strategic approaches _{189 10.2% 102.9 8.2%} Urban environment _{69 3.7% 35.4 2.8%} Nature _{771 41.7% 637.2 50.9%} Habitats _{580 31.4% 494.6 39.5%} Birds _{181 9.8% 139.6 11.1%} Other _{10 0.5% 2.9 0.2%} Total

_{1,847 100% 1253.1 100%}

Source: BUTLER database. Note that the total here is smaller than in Table 3.5 above as this table only covers NAT and

ENV projects (not TCY).

3.2.2

Types of beneficiaries

For the purpose of this evaluation, we have distinguished between three main types of beneficiaries:

Public entities; NGOs and research institutions; and public and private enterprises. Table 3.6 shows that

public entities are the most typical beneficiary accounting for 44% of all projects. However, public

entities are particularly important in relation to Nature and Third Country projects and less so in relation

to Environment projects, where the beneficiary is more often a public or private enterprise. More

detailed tables on the specific type of beneficiary can be found in Appendix 7.

Table 3.6

Beneficiaries of LIFE Projects, 1996-2006

Beneficiary type LIFE Environment LIFE Nature LIFE Third Countries

Total

Public entities 360 (33%) 448 (58%) 92 (51%) 900 (44%)

Public and private enterprises

429 (40%) 19 (2%) 14 (8%) 462 (23%)

NGOs and research organisations

194 (18%) 226 (29%) 58 (32%) 478 (24%)

Not specified 93 (9%) 78 (10%) 15 (8%) 186 (9%)

Total 1076 (100%) 771 (100%) 179 (100%) 2026 (100%)

Appendix 1

Indicators for assessment of results and impacts

During the inception phase, we have worked on identifying indicators to measure effects (results and

impacts) at the thematic level, which can be applied to the projects analysed and which can be used as a

basis for aggregation from the project level to the thematic level. We have done this through:

•

Reviewing the definition/objectives for the themes, as set out in the LIFE+ Regulation compared to

the LIFE II and III regulations.

•

Reviewing a number of project summaries to trace type of result and impact for typical projects

under the various themes.

•

Reviewing the 6th EAP, the related thematic strategies and EU regulatory documents

•

Reviewing the GHK Study on the Establishment of Indicators to Assist Monitoring of Measures

Financed by LIFE+

•

Reviewing indicators suggested by the European Environment Agency (Core Set of Indicators, ref.

Technical Report No. 1, 2005 and Halting the Loss of Biodiversity, ref. Technical Report No.

11/2007)

Our primary focus has been on developing indicators, which enable the measurement and subsequent

aggregation of effects from project level to programmatic level. Where relevant, this has been

supplemented by other types of indicators focusing on a more indirect assessment of results and

impacts. I.e. most indicators aim for a quantification of impact, but in the cases where this was deemed

impossible, relative indicators using, for example, a scale from 1 to 5 are suggested.

The indicators developed are divided into two sets - one set pertaining to the results of the projects

under each theme, and one set pertaining to the impacts of the projects. By comparison the GHK study

refers to three types of effect indicators: results, outcome and impact, where result indicators refer to the

immediate effects, outcome indicators refer to intermediate effects and impact indicators refer to longer

term effects

. For the purpose of this evaluation we do not find it realistic to distinguish between

intermediate and longer term impacts. This may be carried out in an evaluation of individual projects but

not in this programmatic evaluation given the large portfolio of projects analysed and the many different

start and finish dates. Rather, we have sought to develop both (short term) results indicators and (longer

term) impact indicators for five different types of effects which address the types of changes mentioned

under effects in the GHK study (ref. figure 3.2, p. 11).

We have identified five different types of effects as listed below.

•

Environment:

Improved state of or reduced pressure on environment, nature and/or human health

•

Management:

Improved management, planning, administration or regulation

•

Awareness:

Increased awareness among a broader range of stakeholders about the environmental

issue in question

•

Demonstration:

"showcase effect" of demonstrating specific models, methods and/or technologies

to potential users

•

Innovation:

Innovation/development of new technology or improvement of processes e.g. to save

energy or other natural resources, cause less pollution or reduce the generation of waste

We have developed theme-specific indicators for environmental and management effects. These are

described below in a separate section on each theme. We have also developed a number of general

indicators which are NOT theme-specific for management, awareness, demonstration and innovation

effects. These are described in the final section at the end of this appendix.

As also stated in the GHK study, environmental effects are likely to occur mostly in the medium to long

term and thus impact indicators seem more relevant than result indicators. Nevertheless, in many cases,

we suggest that the indicators for measuring both results and impacts be used, as some projects may

have generated environmental effects even in the short term and these should be documented by the

evaluation. It will then also be possible to compare short term results with long term impacts and,

hopefully, illustrate extended impact over time and the sustainability of the projects.

Below, we first present theme-specific indicators for the themes in the following order:

•

LIFE NAT (one common set of indicators developed to cover the four types of conservation actions

according to which the analysis of LIFE NAT will be structured)

•

LIFE ENV

-

climate change

-

air

-

water

-

soil

-

forests

-

natural resources and waste

-

chemicals

-

urban environment

-

strategic approaches

Finally, we present general indicators related to management, awareness, demonstration and innovation

effects.

Indicators for results and impacts under LIFE NAT

The NAT component of the LIFE Programme has existed in all the generations of LIFE being evaluated

in this ex-post evaluation. This component's focus has been relatively consistent throughout time period

covering the implementation of the Habitat Directive and the Birds Directive. Indicators have been

developed accordingly. As described in Chapter 3, the guiding principles for categorisation of NAT

projects have not yet been settled. It is assessed that results and impact indicators for the NAT projects,

as a group, will remain applicable no matter which guiding principles for categorisation are applied.

Table 1

Result and impact indicators for LIFE NAT

LIFE Nature

Examples of typical project issues Result indicators Impact indicators

Projects contributing to implementing Community policy and legislation on nature and biodiversity, in particular Directives 79/409/EEC and

92/43/EEC.

Supporting the further development and implementation of the Natura 2000 network.

Environmental:

Habitats restored/managed (area km2; no. of priority habitats)

Favourable conservation status prepared for species / habitats (no. species listed on directive annexes/ no. habitats / no. priority habitats)

Species reintroduced (no. individuals)

Invasive species controlled (area / no. of priority habitats)

Management

Management plans agreed (area coverage / partnerships / funding source identified)

Partnerships established

Land use agreements established Land purchase and land

compensation measures conducted

Enlargement of national Natura 2000-network under preparation Improved species protection measures

Other regional / national initiatives within nature conservation Plan for monitoring system for biodiversity in the managed area Transnational cooperation established

Recreational facilities established in the project area improving visitors' numbers/ awareness of area characteristics. (no of visitors)

Environmental:

Habitat management agreed (area km2)

Favourable conservation status achieved for species / habitats Species surviving (population size / no. individuals)

Invasive species controlled (area / no. of priority habitats) and no. of recurring actions reduced

Species population growth (% / no.)

Management

Management plans in operation Partnership agreements maintained and adhered to in management Habitat management agreed, based on non-LIFE funds

Land use developing according to plan

National policy a nd regulatory change (orders, local/regional conservation initiatives)

Recurring fund-raising needs reduced

Enlargement of national Natura 2000-network achieved

Regularly recurring monitoring on species diversity and number in the protected/managed area

Transnational cooperation under execution

Biodiversity monitoring system in place and under operation

Indicators for results and impacts for LIFE ENV: Climate change theme

Climate change is a new theme which has been included in LIFE + programme, i.e. there are no specific

references to a climate change theme in the preceding LIFE regulations (LIFE II and LIFE III) although

"reduction of emission of gases having a greenhouse effect" is mentioned as an objective under LIFE III

and "tropospheric ozone" as a sub-theme in LIFE II. We have chosen to distinguish between the projects

that fall under the climate change theme from those under the air theme using the general principle that

climate change projects relate to physical or chemical processes and impacts occurring in the upper

atmosphere, while the air theme relates to air pollution exhibiting its effects on or near ground level.

Hence, project types to be categorized as climate change projects include those, which aim to eliminate

or reduce emissions of greenhouse gases from a large variety of sources including energy production,

road, railway, air and water transport, different industrial processes, domestic heating, agricultural

production including husbandry etc. and similar projects regarding ozone depleting substances (ODS).

Furthermore, projects which aim at adaptation to climate changes and mitigation of the negative effects

of such changes (e.g. flooding) are categorized as climate change projects. Finally, projects developing

models and other systems to predict and/or assess climatic changes and their consequences, or

developing/introducing managements systems (e.g. CO2 trading schemes), are considered climate

change projects.

Table 2

Result and impact indicators for climate change theme

LIFE Environment - Climate Change

Examples of typical project issues Result indicators Impact indicators

Reduction of greenhouse gas emissions

Reduction of emissions of ozone depleting substances (ODS)

Adaptation to climate changes including prevention/mitigation of negative impacts such as flooding Prediction of climate changes and their effects by use of models etc. Greenhouse gas emission trading and other climate change management systems

Environmental:

Reduction in emissions of CO2 or other greenhouse gases

(tons/year)

Reduction in emissions of ODS (tons/year)

Size of area(s) (km2) protected against adverse effects of flooding

Number of people protected against adverse effects of flooding

Management:

Climate strategy model used for early warning.

Number of European Emissions Trading Schemes established Monitoring system implemented. (No, yes at national level, yes at international level)

Environmental:

Reduction in emissions of CO2 or other greenhouse gases (tons/year)

Reduction in emissions of ODS (tons/year)

Size of area(s) (km2) protected against adverse effects of flooding

Number of people protected against adverse effects of flooding

Management:

No. of EU indicators on climate change meeting strategy level Number of European Emissions Trading Schemes established Number of EU countries with standardised and compatible monitoring schemes

Indicators for results and impacts for LIFE ENV: Air theme

Air/air pollution has been a recurrent LIFE ENV theme, dealing with bulk air pollutants (SO2, CO, NOX,

VOC etc.) as well as specific hazardous substances (e.g. dioxins) dispersed through the air (lower

atmosphere), their effects on humans and the environment and measures to deal with these issues.

We have chosen to distinguish between projects, which fall under the air theme from those which fall

under climate change using the general principle that the air theme relates to air pollution exhibiting its

effects on or near the ground while climate change projects relate to physical or chemical processes and

impacts occurring in the upper atmosphere. This means that projects on greenhouse gases and ozone

depletion issues are categorized as climate change projects while acidification caused by air pollutants

and ozone as a respiratory irritant are categorized under the air theme.

Air projects include development of technologies/methods to prevent or reduce emissions of general or

specific air pollutants, measures to minimise the negative impacts of air pollution on human health, the

state of the environment or natural resources, and approaches/systems/methods to manage or assess or

monitor air pollution in general or that of specific hazardous substances.

Table 3

Result and impact indicators for air theme

LIFE Environment - Air

Examples of typical project issues Result indicators Impact indicators

Emissions of main gaseous air pollutants ((SO2, NOX, VOC etc.) from industrial sources, power production, traffic etc.

Emissions of specific hazardous air pollutants (e.g. dioxins) from industrial sources

Air pollution from agricultural activities Effects of air pollutants on health

Effects of air pollution on the state of the environment including acidification and eutrofication.

Methods, models etc. for assessment and/or monitoring of air pollution, Management and other regulatory measures to reduce air pollution and its impacts

Environmental:

Number of persons affected by the project's improvement of air quality

Increase in area (km2) with ambient air quality meeting EU air quality standards (or increase in population living in such areas) Reduction (tons/year) in

emissions of noxious gasses (e.g. SO2, NOx, NMVOC an NH3)

Reduction in use of private cars (annual average day traffic)

Improvement of ecosystem negatively affected by acidification

Management:

Monitoring system established

Early warning system established

Environmental:

Number of persons affected by improvement of air quality

Increase in area (km2) with ambient air quality meeting EU air quality standards (or increase in population living in such areas)

Reduction (tons/year) in emissions of noxious gasses (e.g. SO2, NOx, NMVOC and NH3)

Reduction in use of private cars (annual average day traffic) Improvement of ecosystem negatively affected by acidification.

Indicators for results and impacts for LIFE ENV: Water theme

Water is a broad theme covering surface and groundwater, freshwater and coastal and marine waters,

water as an environmental (ecological) compartment and as an essential resource etc. It has been a key

ENV theme through the different stages of the LIFE programme.

Projects to be categorized under the water theme include development and implementations of river

basin management plans and similar schemes for other water systems, technologies and methods to