I MPLEMENTATION ASPECTS ICT-component

In several projects an ICT-component is included. It seems that this component is not always a success:

− In the Czech Republic the originally scheduled ICT-component was abandoned.

− In Slovakia it is not clear what kind of effects the purchase of equipment generated.

− In Poland the project was for example seen as an ICT-project by some involved public officers, whereas the overall aim went beyond that. In this case no one verified whether higher-level objectives were achieved.

One may also question whether investments in equipment are additional in the specific situation. It is often very difficult to answer this question. Another aspect is that the equipment should be replaced regularly and it is not likely that EC-funding can be used for each investment needed.

Delays in preparation and approval of the project

This applies especially to Poland I (environment) and Poland II (crime), but also to Slovakia and Hungary (ESF). In Hungary the delay was caused by the fact that the capacity building is part of a larger programme. This larger project is managed by an organization that had to delegate responsibility for specific subprograms (of which the capacity building project is one) to other organizations. The overall managing organization and the organizations responsible for the different components of the capacity building held different views on this project. This led to a long preparation period. At the same time only little time was spent on the intervention logic and on identifying training needs (training is the core activity in this project). In Poland II a lot of time passed between the design of the project fiche and project approval and, hence, actual implementation. Meanwhile many rapid changes happened in the environment which determined that in the end some of the projects results turned out to be little. Unclear role of training/workshops in the projects

This applies to SR, PL and but also to a certain extent to CZ. The trainings and workshops are highly valued by people, but the question remains what it really produced/delivered. This is also not clear from evaluations/reports. Sometimes (such as in the Hungarian (ESF) case) interviewees indicated that monitoring results proved the satisfaction of the trainees, but could not provide tangible evidence.

Activities seem not to be the best ones in respect of the problems/context:

In some cases the activities chosen (or some of them) were not the most appropriate ones to reach the objectives. Examples are:

− Poland I: other activities would probably have led to better results.

− Portugal: apparently a mismatch exists between the problems and the kind of training chosen.

− Czech Republic: the relevance of the development of the central database for securities is unclear. The database was also not actually realized.

− Greece Measure 2.5: there were problems with formulating and identifying relevant and proper activities.

It is likely that this due to vague objectives (Poland I and Portugal) and to a poor intervention logic (all three cases).

Active involvement of beneficiaries in the project (participative approach)

A participative approach was particularly used in Cyprus, Slovakia and the Czech Republic. However, in the Czech project the National Central Bank was mentioned as an important missing organization in the project. This omission might have been avoided with a better-designed intervention logic.

Indicators of achievement often of poor quality or not used in practice

In most projects the use of indicators to monitor or evaluate outcomes is unsatisfactory. The following situations can be identified:

− Many indicators are defined that are actually not used (Slovakia).

− Indicators are included later (Poland).

− Indicators change over time (Cyprus).

When this applies to the overall and global objectives, it causes little harm to the project. That is because it is often impossible to assess the impact of the capacity building project on the degree to which these overall objectives are reached. Let us take the example of the Czech case. By improving the legislation with respect to securities one hopes to improve the functioning of the capital market and ultimately the pace of economic growth. Clearly, the latter is affected by so many factors that it will not be possible to assess the influence of the capacity building project on economic growth. So, it does not make sense to specify in this project quantitative targets with respect to the effect on economic growth. It is sufficient that in the ex-ante phase of the project an analysis is made on the basis of the theoretical and empirical literature, which indicates that proper legislation is crucial for a well functioning capital market and that the latter is an important determinant for economic growth. It makes more sense to specify measurable targets with respect to outcomes that are more closely connected with the project’s outcomes. And these targets can and should then be monitored and evaluated during implementation. This in turn provides feedback that may help to improve implementation. The lesson to be learnt is that a limited set of indicators (smart) should be used that are closely connected to the direct output of the project and are actually used during implementation.

Furthermore and connected to the point of the indicators: there are often weak monitoring mechanisms in the projects. The outcomes of the monitoring are at least not well documented. Furthermore, evaluation is often poor. Partly this is connected to the poor monitoring systems that do not provide the information necessary for evaluation.