Impacts

The survey on the use of and satisfaction with HIT and the focus group provide sound evidence that HIT has benefitted programme authorities and applicant/beneficiaries alike. HIT has helped increase the efficiency and effectiveness of programme management in a substantive number of programmes, not least because it enabled the development of the eMS, but, above all, because it has led to a more uniform interpretation of regulatory requirements across programmes, increasing legal assurance. For applicants and beneficiaries, it has positively affected the efficiency and effectiveness of applying for project funding and implementing a project, especially for those working in multiple programmes.

Table 2: Summary of impacts found

Impact Definition Observation

Efficiency Achieve more with less, e.g. through optimising service delivery

 HIT contributes to a more efficient programme implementation as it offers programmes off-the-shelf implementation tools in line with EU regulations and guidance, sparing them from having to develop tools on their own.

 HIT achieves simplification for actors (applicants/beneficiaries, national controllers, national authorities) working in multiple programmes.

Effectiveness Doing the right

thing to increase quality and performance, e.g. transparent decision-making, staff development, better communication

 HIT increases legal certainty for programme authorities resulting from the large number of HIT adopters and the more harmonised interpretation of regulatory requirements.  HIT is one building block in the development of a common

Interreg brand identity, contributing to the awareness of Interreg and its achievements.

 HIT fully incorporates the focus on results and support programme performance through results delivery.

Resilience Ability to deal with and manage change

 HIT achieved that programmes jointly address challenges arising from a changing regulatory environment.

4.4.1 Increased efficiency

Resource savings in programme authorities. The development of the HIT was a time-consuming

and resource-intensive process for the programmes that actively contributed to it. Nonetheless, the survey and focus group have provided plenty of (anecdotal) evidence that programmes were able to reap savings in terms of time and staff resources:

 80.8 percent of survey respondents agreed or rather agree that HIT saved staff resources as programmes do not need to develop their own implementation tools.

 Especially for programmes that either did not exist in the previous programming period or whose start was considerably delayed, HIT meant that there were tools readily available

when they needed them, which saved them time and helped them catch up.

 Since the ambition was to develop harmonised tools that are based on minimum

requirements regarding legal compliance and good programme management practices, HIT

promoted a debate on what data are really needed for assessing, monitoring and controlling projects. Even though the principle of simplification was not always strictly followed as more

and more optional elements were added over time, programmes still understood HIT as an opportunity to pick up good practices from other programmes for the sake of simplifying their own practices and tools.

 The main efficiency gain of all is probably the fact that HIT enabled the development of the

eMS. Around 40 percent of Interreg programmes are using the eMS and hence benefit from

the fact that it has saved from having to develop a monitoring system on their own (see chapter on the evaluation of eMS).

“HIT enabled us to finish the designation process in early 2016, due to the ready- to-use templates and procedures.” Survey respondent from IPA CBC Programme The effect of HIT on simplification is sometimes reduced by gold-plating on Member State level. Various programmes reported that they had to make changes to the forms to comply with national requirements. One focus group participant, for example, reported that her programme had to add a lot of additional checks to the FLC forms due to the rigorous and detailed national audit procedures.

Simplification for actors working in multiple programmes. In addition to programme authorities,

other programme stakeholders also benefitted from the fact that HIT reduced administrative burden. 84.6 percent17 _{of respondents to the HIT survey agree that HIT makes life easier for}

applicants/beneficiaries working in multiple programmes. Programmes report that those

applicants/beneficiaries that are involved in projects in different programmes find it easier to handle the different programme forms as they are very similar and applicants/beneficiaries can handle them better than before they were harmonised.

The satisfaction of applicants with the application forms was also rated quite highly by programmes (See Figure 14).

“We find that applicants from countries that don’t use HIT need a lot more training than those who have seen the forms before in other programmes.” Participant at HIT focus group

80.8 percent of respondents to the HIT survey believe that the harmonised tools for financial control and audit support national controllers working for multiple programmes. However, they rate the satisfaction of controllers with the tools considerably lower (53.9 percent).18 _{One shortcoming of the}

process in this respect was that members of the working group were usually from the JS (or MA) and often did not have a clear mandate to speak and decide on behalf of national control bodies. In the future, it would be good to collect feedback from end users more systematically in order to develop tools that are user friendly.

Lastly, the harmonised tools led to simplifications for national authorities as they receive similar forms and information from all programmes in which they are involved.

17 _{These responses are partly based on the perceptions of survey respondents, but some programmes also have}

carried out surveys to applicants/beneficiaries.

18 _{Note that a similar survey on eMS has revealed a high satisfaction of both and national controllers with eMS,}

which use the harmonised tools. Applicants/beneficiaries and national controllers were not surveyed for this evaluation. Presented figures are mainly based on the perceptions of programme managers and should be treated with caution.

Figure 14: Satisfaction of applicants/beneficiaries and FLC with the tools

Source: HIT survey, 2018. N=28 4.4.2 Increased effectiveness

Impacts on the effectiveness of programme management. Besides the impact on the efficiency of

programme management, HIT also positively affected a number of factors associated with the quality of programme management such as increased legal certainty for programme authorities, improved results performance or the positive contribution to the creation of an Interreg brand identity.

There is sound evidence that HIT has increased legal certainty for programme authorities. 88.5 percent of survey respondents agree that HIT provided assurance that they are using implementation tools in line with the EU regulations and guidance. Statements from programmes in the survey and focus group also testify that increased legal certainty was an important reason for them to use HIT.

“The Harmonised Implementation Tools make us feel that we are on the safe side.” Survey respondent from Interreg CBC Programme

Certainty has increased as a result of:

 the more uniform interpretation of regulatory requirements (reducing “lonely programme solutions”);

 the decreased likelihood of missing out on anything as “many eyes are on it”; and

 the fact that a majority (‘critical mass’) of programmes are using HIT, which makes it hard for auditors to question the regulatory compliance of the tools.19

The uniform interpretation of regulatory requirements among programmes is a major achievement of the HIT process. The exchange taking place in the scope of the HIT development has very much shaped how programmes have interpreted the newly introduced requirement on results orientation and how they have translated it into their implementation tools. Programmes confirm that the clear and harmonious approach to results-orientation helps them report on outputs and results. One programme even reported that it led to more clearly focused projects.

“In our programme, the quality of the project has substantially risen as a result of requesting projects to have a clear intervention logic in the Application Form.” Survey respondent from Interreg CBC Programme

19 _{Nonetheless, one programme reported that compliance was questioned by the programme’s auditor in spite of}

the programme using HIT and eMS.

7.7% 15.4% 15.4% 61.5% 50.0% 38.5% 3.8% 7.7% 26.9% 34.6% 38.5% 0% 20% 40% 60% 80% 100%

Satisfaction of applicants using the Application Form.

Satisfaction of beneficiaries using Project and Partner Progress Report templates.

Satisfaction of national controllers using FLC certificate, FLC control report and FLC checklist.

Evidence from one programme shows that the harmonised application forms allow the identification of

potential overlaps with and double funding of projects from neighbouring programmes.

“Having harmonised application forms helps cross-checking with other programmes the potential existence of overlaps/double funding.” Survey respondent from Interreg TNC Programme

And lastly, HIT has also had an impact on the visibility of Interreg. 96.1 percent of respondents agree that HIT has positively contributed to creating an Interreg brand identity. HIT is one building block in the overall endeavour to increase the awareness of Interreg.

Figure 15: Impacts of harmonised implementation tools

Source: HIT survey, 2018. N=28

4.4.3 Increased resilience

The collaborative development of harmonisation implementation tools meant that programmes jointly address challenges arising from a changing regulatory environment. The process enhanced self- reflection, and organisational flexibility, improving change management in programmes, which contributes to more resilient programmes.

61.5% 65.4% 46.2% 53.8% 50.0% 65.4% 57.7% 42.3% 11.5% 19.2% 34.6% 38.5% 30.8% 23.1% 42.3% 53.8% 7.7% 11.5% 11.5% 7.7% 11.5% 11.5% 3.8% 19.2% 3.8% 7.7% 7.7% 0% 20% 40% 60% 80% 100%

HIT provided the basis for the development of a common monitoring system (eMS) for Interreg programmes. HIT makes life easier for applicants & beneficiaries working in

multiple programmes.

HIT simplified the work for control and audit bodies working with multiple programmes.

HIT development provided an opportunity to exchange in depth with other programmes over their programme implementation

tools.

HIT saved staff resources as we needn't develop our own programme implementation tools.

HIT assured us that we are using implementation tools in line with regulations.

HIT is based on good practices from across the community of Interreg programmes.

HIT contributes positively to creating an Interreg brand identity.