Support/involvement

The e-survey revealed that most of the stakeholders are familiar with the EU public health policy in general. This also holds for the general programme ob- jectives and annual priorities of the PHP but to a somewhat lesser extent. In general, beneficiaries feel more familiar in this area than other stakeholders. However, other stakeholders employed by international organisations are also very familiar with the EU public health policy and the way the programme sup- ports this policy. Stakeholders employed in the public administration of the Member States feel less familiar in this area. This is an important observation as familiarity is considered to be closely associated with involvement.

Most beneficiaries have met barriers to receiving funding. Possible barriers in- clude language problems, procedures and cultural differences. As an example, requirements to management might be difficult to meet by some PHP appli- cants as pointed out by Commission staff interviewed as part of the evaluation. Furthermore, some stakeholders might have problems finding the supplemen- tary funding necessary to participate in the programme.

The needs of the different Member States may be translated into priorities in the annual work plans (AWPs), activities selected for funding and into in- volvement in the implementation of the funded activities. The Commission, the Member States and the beneficiaries all have important roles to play in this re- gard.

The implementation of the programme should promote national involvement at all levels, including actual involvement of Member States in the choice of pri- ority areas for the annual work plans (AWPs). This is important to increase the potential use of project output and results at national level. Furthermore, it is important that the Commission raises awareness among national stakeholders that complementary funding is highly supportive. The introduction of joint ac- tions as a new financial instrument with the second Health Programme 2008- 2013 is a step in this direction.

Through participation in the Programme Committee, the Member States have the opportunity to influence the implementation of the programme. The Pro- gramme Committee has to give its opinion on the implementation measures defined and decided by the Commission, including the annual work plans (AWPs), selection criteria and financing of actions. According to Commission staff interviewed as part of the evaluation, the actual participation/involvement of Programme Committee members differs across countries depending on im- portance attached to the programme by national systems and individual factors. In general, Programme Committee members do not seem to consult operating stakeholders at the national level to a sufficient degree. Furthermore, the fre- quent turnover of Programme Committee members tends to reduce participa- tion/involvement by the country in question.

The case studies point to good examples of projects which ensure participation at the national level, e.g. through appointment of national coordinators with When looking at

support/involvement, we conclude that most stakeholders are familiar with EU public health policy and the PHP …

… most beneficiaries have met barriers to receiving funding …

… and the degree to which the needs of Member States are met depends on their participation in the PHP

countries. However, no evidence has been found of the application of project output at the national level. Neither is evidence found of the incorporation of national interests in the implementation of the projects.

On this background, we recommend that:

10 EAHC and DG SANCO should pursue inclusion of Member States which appear inactive in the programme. These are typically countries with a rel- atively low GDP/capita. Inclusion could be pursued by providing technical assistance to write proposals (EAHC) or by increasing the EC financial contribution (DG SANCO), possibly on the basis of an alternative cost model.

The suggestion is based on earlier evaluations of similar programmes e.g. the framework programmes also involving participants from Eastern Europe con- clude that these participants in particular have difficulties in providing co- funding which have led to underrepresentation of this geographical area. It was also highlighted by both Commission staff interviewed as part of the evaluation and in the case studies that small organisations especially from East- ern European might not have the resources necessary to participate in the pro- gramme as it is today. One way to solve this problem could be to increase the maximum co-financing for up to five selected Member States. The five Mem- ber States in question should be selected on a yearly basis and announced in the annual work plan based on strict criteria, e.g. the Member States with the low- est participation level in the programme over the last three years combined with a GDP per capita and/or average life expectancy below EU average.

Today, the financial contribution cannot exceed 60 per cent of the project costs. However, a maximum co-financing of 80 per cent of the eligible costs could be financed if the project has significant European added value. The distinction between a project with significant European added value and a project without European added value is subtle - even more so as there is no clear-cut definition of European added value.

11 EAHC should distribute an information package with relevant targeted in- formation about the programme to each Programme Committee and Na- tional Focal Point members.

The e-survey and the interview with National Focal Points revealed that a con- siderable number of stakeholders employed in the public administration of the Member States are not familiar with the programme. Combined with the fre- quent turnover of Programme Committee members this indicates a need for compilation of an information package, which can be sent by e.g. e-mail to Programme Committee and National Focal Point Members.

Brief descriptions of project results disseminated to national stakeholders at political level (see recommendation 16), including Programme Committee and We recommend:

Higher co-funding of certain costs …

… information pack- age …

12 EAHC should encourage that annual information days are still held at both EU and national levels to increase familiarity with the programme and an- nual priorities.

In 2010, there has not been any general information day at the EU level in Lux- embourg or Brussels as in previous years. National information days on the calls for proposals under the second Health Programme were held in ten of the 27 Member States in late 2009 or early 2010. By committing all Member States to host national information days, Programme Committee and National Focal Point members will have the opportunity to become more familiar with the programme and potential applicants at national level.

13 Each Member State should establish a help desk to provide support to po- tential applicants to overcome barriers relating to funding procedures and reporting.

Our analyses show that it is a widespread experience among project applicants that it is cumbersome process to produce an application and among project co- ordinators to comply with the reporting requirements. According to the EAHC homepage, there are already help desks in a few countries. These should be strengthened and the experience expanded to all countries.

12.5

Monitoring

Monitoring is a continuous and systematic process carried out during an inter- vention that generates quantitative data on the implementation of the interven- tion but usually not its effect. The intention is to correct any deviation from the operational objectives and thus improve the performance of the programme as well as facilitate the subsequent evaluation.

Monitoring reports based on common management performance indicators are not compiled. Issues covered in such reports include e.g. the promotion of the programme (number of information days held and number of attendees), per- formance of the calls (success rate by priority area, funding scheme etc.) and performance of the proposal evaluation (overall quality assessment of the pro- posals, time to contract/grant etc.).

The resources allocated to the monitoring system of the PHP at Commission level must be in line with the financial scope of the programme, which is con- siderable smaller than the framework programmes for research and develop- ment. At the same time, the monitoring system must be designed to avoid im- posing too heavy administrative burdens on the beneficiaries.

Progress has been made since the launch of the PHP to ensure that the monitor- ing system delivers the information needed to support sound implementation of the programme. In our view, there is still room for improvement. During inter- views, Commission staff held the view that more resources should be allocated to the monitoring of the programme. A vast amount of information is collected … information days

- both at EU and na- tional level …

… help-desk at na- tional level

When looking at the monitoring of the PHP …

… we conclude that progress has been made, but that there is still room for im- provement

including priority area, country, organisation type, estimated starting date, dura- tion and output indicators for monitoring purposes and result indicators for measuring performance. Furthermore, the beneficiaries are required to compile a final technical implementation report describing the process and deliverables produced. Relevant information on the activities funded should be registered in a database in order to ease the monitoring of the implementation of the pro- gramme, including the coverage and results of activities funded - in line with the above discussion of the need for output indicators. Based on this register, regular reports on the implementation may be produced and distributed to rele- vant stakeholders.

On this background, we recommend that:

14 EAHC should compile monitoring reports on a yearly basis based on common management performance indicators.

During this evaluation we did not find a systematic monitoring effort. We sug- gest that monitoring reports should cover issues related to promotion of the programme, performance of the calls and performance of the proposal evalua- tion. The monitoring report should provide a clear picture of the effectiveness of the programme implementation and point to possible gaps whereas the level and quality of output are to be reported elsewhere. It is suggested to draw inspi- ration from the monitoring reports of the framework programmes for research and development taking into account the principle of proportionality, i.e. the different financial scope of the programmes.

15 EAHC should predefine keywords for the categories of interventions, health issues and the target groups. The project applicants must choose the keywords which best describe their projects. This improved information about coverage of health objectives will enhance both funding decisions and evaluation exercises.

Today, the beneficiaries have to choose priority area, action and sub-action when applying for funds under the second Health Programme. Furthermore, they have to describe verbally the contribution of the project to the programme and annual work plan (AWP).Our experience from the Portfolio analysis dem- onstrates that it is cumbersome to extract such keywords from the project ab- stracts. Furthermore, we also envisage that applicants in the future may choose more than one priority area as projects may cover more than one area, e.g. by choosing primary, secondary and other priority area. Finally, we propose that applicants choose intervention type, health issue and target group in the same way, e.g. primary, secondary and other.

We recommend:

A more detailed mapping of activities available for data- base extract