Summary of the ex-ante evaluation of the ERDF programme 81 Ex-ante process and method

This report contains the final assessment of Denmark’s draft programme for the ERDF (national programme for the ERDF). The report is based on the final draft programme dated January 2014. As part of the preparation process, a public consultation process about the programme has been held, the results of which are reflected in the programme.

The ex-ante process has been characterised by the interplay of comments on the draft programme (including assessments and recommendations for specific topics) and meetings at which these have been discussed. The assessment criteria and method are based on ex-ante guidelines issued by the European Commission.

The strategic environmental assessment (SEA) was carried out by a separate team of environmental experts and closely coordinated with the ex-ante evaluation. A public consultation about the strategic environmental assessment has been held. The public consultation did not give rise to changes concerning the strategic environmental assessment.

Evaluation of the programme strategy, appropriateness and needs Challenges and needs in relation to Europe 2020

The ex-ante evaluator assesses that there is good continuity between the different analyses of the challenges in the Danish economy, including in the European Commission’s position paper and the analyses carried out in the Partnership Agreement. The evaluator also assesses that:

- there is general agreement about the most significant challenges: increased productivity, competitiveness and the labour supply, job creation and efficient use of energy and resources;

- there is a convergence between the conclusions of the above-mentioned analyses concerning possible actions;

- account has been taken of the siting: each growth forum can consider the region’s unique strengths, weaknesses and development needs.

The ex-ante evaluator generally assesses that the analysis covers the programme area and its needs, and forms the basis for selecting priorities and action areas. The recommendations of the ex-ante evaluator concerning description and line of reasoning in the background and rationale for the action areas chosen (innovation through cooperation, the description of the challenges associated with efficient use of energy and resources by SMEs, and sustainable green urban development) have been strengthened in the latest draft programme. The most significant needs and challenges relating to Europe 2020 are presented in the Partnership Agreement. The ex-ante evaluator assesses that these are well-reasoned and illustrated in the relevant documents.

The ex-ante evaluator’s recommendation that more recent data should be used has largely been followed. The figures for public-sector – and private-sector – investment in R&D have been updated. It is assessed that there is consistency between sources, analyses and choice of actions. Based on the description of the preparations for the programme document for the ERDF (as well as the Partnership Agreement), it is assessed that the Partnership (relevant ministries, regional authorities, as well as representatives of management, labour, business organisations, etc.) has been involved at several stages of the programme-development process. The ex-ante evaluator has not taken part in this process, however, and for this reason the process can only be assessed on the basis of the description in the programme document.

Alignment between programme objectives and challenges

The programme document reviews each thematic objective and the investment priorities selected in the light of the challenges facing Denmark. It is generally assessed that the objectives are well- aligned with the investment priorities chosen.

The ex-ante evaluator notes that the European Commission guidelines for ex-ante monitoring and evaluation of the ERDF open up the possibility of specific objectives not having to be quantitatively measurable. The results for all priorities have now been made measurable by means of result indicators. The ex-ante evaluator notes that this can be difficult to determine accurately, and that it can be difficult to determine the extent of the contribution to a programme-specific action, in relation to the overarching Europe 2020 objectives.

It is generally assessed that the programme’s formulated objectives contribute to Europe 2020 and address national and regional challenges. It is assessed that the programme is highly consistent with the relevant policies and strategies, and that the programme contributes to Denmark’s objectives and actions. The programme complements in particular the Growth Plan for Denmark (the Danish Government’s growth plans for areas in which the Danish business community is internationally competitive), Denmark’s innovation strategy and the Green Conversion Package. This also includes the Strategy for Denmark’s Cluster Policy and networking actions.

It is generally assessed that the additionality principle is respected, as the regional funds are used to support existing Danish policies and strategies. It is assessed that the programme generally leads to action/projects that would not otherwise have been initiated to the same extent without EU assistance.

The Partnership Agreement refers to the evaluations of the existing programme which provide good indications of the new programme development process. A recommendation of the ex-ante evaluator to detail lessons learned in relation to types of action in the description of priority axes has been followed.

It is assessed that the programme is aligned with the regulation and the common strategic framework (Annex I of Regulation (EU) No 1303/2013 of the European Parliament and of the Council of 17 December 2013). It is assessed that the breakdown complies with the regulation’s requirements, as 80% of the funds are allocated to three thematic objectives: 1, 3 and 4. In all, 20% is allocated to thematic objective 4.

Most of the funds are allocated to priority axes 1 (promotion of innovation) and 2 (promotion of SME competitiveness), as the need is deemed greatest in these areas. In addition, these priority axes include familiar activities where lessons have been learned in previous programmes and where the demand exists from the outset. Thus, fewer funds have been allocated to the new priority axes focusing on efficient use of resources. This is assessed as being a realistic strategy in accordance with the additionality principle, as substantial funds are allocated to green actions in other Danish initiatives.

The fixed milestones are scheduled for 2018, where 25% of the funds are expected to have been used and 25% of the objectives are expected to have been achieved. These figures are based on lessons learned from previous programmes and are deemed realistic, although conservative, by the ex-ante evaluator. Under priority axis 4 (sustainable urban development), implementation is expected to be faster, as the actions are based on a few pronounced rounds of applications.

Internal and external coherence of the programme External coherence

The groundwork for synergy is laid in several areas between the actions under the ERDF and the ESF. The ESF contributes, inter alia, to accommodating enterprises’ need for growth-orientated skills enhancement in areas where an enterprise needs expertise, so that more knowledge in this area is brought out to the SMEs. In addition, entrepreneurs who are off to a good start in relation to the action under the ESF could benefit from the action under the ERDF for SMEs with growth potential.

Generally speaking, it is assessed that the programme complements the EU programmes COSME, LIFE and HORIZON 2020 by focusing on enterprise actions, whereas the EU programmes focus on the framework conditions for SMEs or transnational collaboration. The programme potentially contributes to EaSI, but there seems to be no convergence with projects eligible for financing under the New Entrants Reserve/NER300 programme.

The ERDF programme makes it possible for projects to be able to receive funding for collaboration within the framework of the EUSBSR. Specific models are proposed for how this can be done, in the event there is an interest in doing so by the project partners and growth forums.

With regard to external contexts and other factors’ influence on the programme, it is assessed that the way in which external factors could influence the programme’s results has not been explicitly explained. In the report, the ex-ante evaluator mentions how possible external factors could influence the results of the actions in conjunction with the assessment of the individual priorities. Assessment of internal coherence and the intervention logic of the programme (the priority axes) Overall, the ex-ante evaluator’s comments are largely adopted in the priority axes. The final version of the programme includes intervention logics illustrating the continuity between activities, outputs and results in the priority axes.

The ex-ante evaluator’s assessments of the four priority axes are that the changes which are to be created within the priorities are achievable using the planned activities. The ex-ante evaluator’s recommendations have been followed for the most part, and this has strengthened the description of action areas, etc. Priority axis 4 on sustainable urban areas is the most recently developed priority axis, and in the latest version of the programme it has been tightened up and strengthened in the relationship between objectives, activities and target groups.

The ex-ante evaluator originally indicated a need for an explicit presentation of the critical assumptions and prerequisites in the priority axes. The evaluator provided suggestions for how to address this, and the proposals are largely reflected in the programme. The ex-ante evaluator previously proposed that priority axis 4 would be stronger if it included a more detailed description of the potential for actions aimed at supporting sustainable urban development, which has now been included.

In several priorities, the choice of support forms is based on previous experiences, and this is now explicitly included in the description and line of reasoning for the selection of action area, forms of support and types. The ex-ante evaluator had recommended that previous experiences be included

wherever possible, particularly for priority axes 1, 2 and 3 (the latter in part). As priority axis 4 and parts of priority axis 3 are new (action areas focused on resource efficiency), previous experiences cannot be included, however.

A decision has been made to focus the action on SMEs in priority axes 1, 2 and 3, which is expected to generate clear synergy and complementarity within the action, planning and information about support options. Whereas priority axes 1 and 2 are focused on increasing the number of high-growth and innovative enterprises, priority axis 3 focuses on the quality/sustainability of SME production and growth. Thus, convergence within the target group can occur under the priorities. In terms of methodology, complementarity exists between priority axes 1 and 4, as both priorities focus on supporting development processes, and between priority axes 2 and 3, which focus on enterprise development in the action phases screening, consultancy and implementation.

It is generally assessed that the formal requirements for the cross-cutting principles equality

between women and men, equal opportunity and anti-discrimination as well as sustainable development are met. As a result of the public consultation, the Danish Business Authority has

brought particular focus to bear, under equal rights and disability, on projects targeting anti- discrimination and equal opportunity, including accessibility. These projects must take precedence during the project selection process.

Indicators, monitoring and evaluation Indicator relevance and clarity

It is generally assessed that the result indicators depict overall the results expected from the interventions. The assessment of the proposed indicators is based on the RACER criteria. The indicators generally meet these criteria.

The output indicators (programme-specific and common output indicators) in the four priority axes reflect the changes resulting from the interventions. For instance, the number of supported enterprises will naturally be affected when support is granted to enterprises. As a new feature, the resource-efficiency indicator has been added to priority axes 3 and 4. The ex-ante evaluator approves of this as an interesting, ambitious initiative that will require the development of data collection by the programme authorities.

Monitoring values are set for all output indicators. For the result indicators, baselines are only set up in keeping with the European Commission guidelines. The ex-ante evaluator has assessed and adopted the monitoring values for all output indicators and has helped to develop these monitoring values particularly for priority axis 3 and in part for priority axis 4. These are deemed realistic and achievable. As regards the resource-efficiency indicators in priority axes 3 and 4, the ex-ante evaluator points out the uncertainties that will arise from insufficient experience, and recommends meticulous follow-up on and possible adjustment of these during the programme.

Control, data sources and data quality have been considered in the programme

Data sources are set up for both the results and the output indicators in the programme document. In most cases, the source of data for the result indicators is Statistics Denmark. The output indicators must be measured and reported on by the beneficiary and/or the project participants. It is

recommended that a set of standard templates/guidelines be developed for the measurement to strengthen this type of data collection.

Evaluations should not be carried out solely on the basis of registry data. The importance of generating data that can be used in evaluation contexts during the period and for qualifying the use of registry data is emphasised by the ex-ante evaluator. For instance, registry data can suffer from the fact that enterprises’ turnover, which is a common analysis variable in terms of impact, is combined with many factors other than those affected by the enterprises’ participation in the project. The effects can be rendered clearer by means of counterfactual evaluations, for instance. The ex-ante evaluator assesses that the implementation mechanisms proposed are acceptable. A large volume of experiential data has been collected in the system and should preferably ensure the efficient implementation of the programme. Institutions taking part in programme implementation and monitoring at national and regional level have extensive knowledge and experience of implementing programmes.

In relation to reducing administrative burdens, it is assessed that initiatives are already under way to reduce administrative burdens. A clear timetable has also been presented for when the development of these initiatives must be completed and they must be ready for implementation.

Contributions to Europe 2020

The programme is assessed as contributing to the following flagship initiatives in particular: ‘Innovation Union’, ‘A digital agenda for Europe’, ‘Resource-efficient Europe’, and ‘An industrial policy for the globalisation era’. Through its general contribution to regional development, the programme indirectly contributes to the objective on territorial cohesion contained in the flagship initiative ‘European platform against poverty and social exclusion’.

The ex-ante evaluator generally assesses that the programme contributes to Denmark’s Europe 2020 objectives on R&D (through priority axes 1, 3 and 4), job creation (through priority axes 1, 2 and 3), and climate and energy (through priority axes 3 and 4).

Strategic environmental assessment

The basic assessment is that the implementation of the action areas under the programme will support the national environmental objectives, primarily through optimisation of resources and energy in enterprises and major urban areas. As the programme is not assumed to have any significant negative environmental impacts, there is no apparent reason to prepare or implement a monitoring programme. An eight-week public consultation concerning the environmental assessment has been held, along with the draft programme, from 9 July 2013 to 3 September 2013. 1.2.4 Summary of the ex-ante evaluation of the ESF programme82

Ex-ante process and method

This report includes an evaluation of Denmark’s draft programme for the ESF (National Programme for the European Social Fund).

The ex-ante process has been characterised by the interplay of comments on the draft programme (including assessments and recommendations for specific topics) and meetings at which these have been discussed. The assessment criteria and method are based on ex-ante guidelines issued by the European Commission.

The need for a strategic environmental assessment (SEA) was screened as part of the combined ex- ante evaluation. The screening was carried out by a separate team of environmental experts and was closely coordinated with the ex-ante evaluation itself.

Evaluation of the programme strategy, appropriateness and needs Challenges and needs relating to Europe 2020

The ex-ante evaluator assesses that there is good continuity between the different analyses of the challenges in the Danish economy for other programmes, including in the European Commission’s position paper and the analyses carried out in the Partnership Agreement. The evaluator also assesses that:

- there is general agreement about the most important challenges to the Danish economy: increased productivity, improved competitiveness and higher supply of labour and qualifications, and the efficient use of energy and resources;

- account has been taken of the siting: each region can consider the region’s unique strengths, weaknesses and development needs. However, the evaluator has recommended that the Danish Business Authority be aware of the fact that consideration for siting on the one hand and taking account of the fact that the programmes must be capable of accommodating changed economic conditions on the other do not impair taking account of the impact. The evaluator has indicated specific needs for clarification and explanations/changes of the analysis of national and regional challenges to growth, which have largely been adopted. Among other things, the evaluator has recommended that, in keeping with the European Commission’s position paper,83 focus be brought to bear on groups with a low level of education, non-European citizens, young people and the elderly, as well as individuals with reduced ability to function in conjunction with the describing of the need to increase the labour supply through increased social inclusion. This recommendation has been followed.

Moreover, the evaluator has recommended that in the description of national and regional challenges, keener focus be brought to bear on the consequences of demographic trends on the labour supply and skill-sets, as well as the consequences of some of the policy decisions taken in 2013 of significance to the supply (early retirement reform), and the demand for labour in the years ahead (infrastructure investments, Growth Plan for Denmark, etc.). The Danish Business Authority has argued in favour of having the programme provide a framework within which to follow up on policy decisions, including to contribute by upgrading the labour-market skills required as part of the many infrastructure investments planned for the years ahead, for instance. The evaluator is satisfied with this.

83 _{Positions of the Commission Services on the development of Partnership Agreement and programmes in Denmark for} the period 2014–2020, Ref.: Ares (2012) 1307708–06/11/2012, European Commission, 2012.

The evaluator has also indicated a number of specific areas in need of further clarification about the challenges to growth, which has also been accommodated.

The ex-ante evaluator generally assesses that the socioeconomic analysis satisfactorily depicts national and regional needs and challenges present in relation to the programme objectives for promoting entrepreneurship and job creation, transnational labour mobility, inclusion through education, and employment and education/training. The evaluator also finds that the programme is well-written and notes that the most relevant aspects of the evaluator’s comments have been incorporated into the programme description.

The evaluator assesses that there is a general need for consistency between sources and analysis,