08 Interventions generally implemented their activities but the extent to which the

produced deliverables contributed to the expected outcomes and impact was not monitored by the Commission which did not identify quantified targets to be reached by its interventions and focused its attention on the monitoring of project and budget support activities and deliverables. (p87)

For budget support, where sector statistical systems do not exist or cannot deliver the data required for result monitoring, the Commission should support the development or improvement of such systems as a matter of priority. (p106) The results obtained by the implementation of reforms supported by the Commission should be monitored on a continuous basis by the Government and the Commission to verify that reforms reach their intended outcomes and that

support has been effective. In addition to the collection on a regular basis of a minimum set of performance result indicators in the sectors supported by SBS, annual sector reviews should be organised between the GoE and the Commission (and other donors) to monitor progress of service delivery performance and, if possible, of use of services by the population. The verification of the contribution to impacts could be undertaken on a less frequent basis. (p107)

CDE 2011 The CDE evaluation concludes (Conclusion 2) that monitoring and evaluation of activities and achievements had been major weaknesses of the CDE, with an approach oriented to the delivery of activities and not to measurement of performance: “The CDE generally did not define clearly the results to be achieved by its activities and did not follow-up closely their performance owing to the absence of monitoring mechanisms: it followed an activity-based approach rather than a results-oriented one.”

The following description is provided:

“Throughout the analysis the evaluation findings pointed to the fact that the CDE did not design its activities with a view to being able to measure their performance and that it clearly failed to follow up and document outputs and results achieved (see Conclusion 1). This issue had already been repeatedly raised in the 2005 Evaluation of the CDE and in the ex post evaluations of CDE sector programmes.

Two major points should be highlighted:

The CDE generally did not define in a sufficiently clear way the expected outputs and results of its activities. In particular:

 in CDE formulation documents, activities, outputs and results have often been mixed up together;

 for its projects or programmes, the CDE mostly set targets in terms of activities to be implemented, less so in terms of outputs, and generally not at all in terms of results;

 when defined, outputs and results to be achieved generally did not embody quantitative targets to be achieved.

Moreover, the CDE did not develop a monitoring system that would have allowed it to follow-up closely the implementation of its activities - in particular to verify whether the activities delivered the expected outputs and results - and to take corrective action during implementation if necessary. In particular:

 Reporting has been weak:

- At the level of the overall portfolio, the CDE has not produced an overall document providing a clear view on all the activities it financed in ACP countries and on the results achieved by them.

- At the level of individual projects, reporting was not systematically carried out. When it was carried out, project progress and final reports were generally produced by the BDS provider or IO or by the beneficiary. The quality and quantity of the information provided varied greatly across activities.

- The CDE created an evaluation form in the form of a checklist to be filled out by the beneficiaries at the conclusion of each project. This checklist, useful in terms of direct feedback from the beneficiary on its satisfaction or not with the services provided, contained very little information on achieved outputs and results.

- There has been no follow-up by the CDE of the development of the performance of its beneficiaries once the activities supported were terminated. This generally gave the beneficiaries the impression that there was a lack of continuity in the support from the CDE. It also deprived the CDE of an important source of capitalisation of know- how since it missed the opportunity of analysing the effects of its activities on the beneficiaries once the support had been provided and of drawing out lessons for future undertakings.

absent:

- An external monitoring system has not been set up.

- As regards programmes, seven sector programmes were discontinued in 2006 and were evaluated at their closure. Programmes that ran throughout the evaluation period were not subject to mid-term evaluations.

- Ad hoc activities have not been evaluated.

This renders complex any assessment of the effectiveness of CDE activities by the CDE itself and by external evaluators. It also precludes a continuous lesson-learning process from taking place within the CDE, a feedback which is essential to enable the CDE to have a clearer picture on the direction to take, to continuously improve its services in line with the needs of its beneficiaries, and to demonstrate its capacity to perform effectively and efficiently.”

Source: ADE (for the European Commission), Evaluation of the Centre for the Development of Enterprise, 2011 (pages 75-76) Findings at JC level - Banking Measures in the Mediterranean area

The report presents the Commission’s monitoring obligations in the case of delegated management assistance: “For delegated management, the Commission is supposed to implement a form of monitoring that is complementary to the monitoring performed by the EIB. The Commission should notably ensure:

 that the banking measures financed under the MEDA programme are adequately managed by the EIB; and

 that information is received by the Commission from the EIB, both at the central and local level, so that it can fulfil its duties.”.

However the report identifies important shortcomings in the Commission’s monitoring of its support to banking measures managed by the EIB: “During the early years of the programme, and until 2005, the EIB did not implement adequate monitoring and controls. In addition the reporting flow from the intermediaries/promoters was insufficient. This mainly affected the monitoring of projects and the implementation of corrective action for projects that were not meeting their objectives or intermediaries/promoters who were not meeting their contractual obligations. This

lack of adequate monitoring affected the Community’s financial interests, notably due to late recovery or non-recovery of funds and the waiving of some contract clauses, for example regarding penalties for late payment.” (p.13).

The report adds that corrective measures have been taken by the EIB since 2005: “Since 2005, when the department responsible was reorganised, the EIB has implemented a more structured monitoring approach, including enhanced controls and reporting, notably with the establishment of individual follow-up sheets » (p.14) and also by the Commission, from 2007, under the ENPI: “Beforehand, it was the EIB’s responsibility under the management agreement signed between the Commission and the EIB to monitor and follow-up the projects funded under budgetary resources. As from 2007, under the European neighbourhood policy (ENPI) budgetary resources, projects managed by the EIB are included in the Commission’s results-oriented monitoring system.” (p.27).

Employment and social inclusion, over 1999-2008

The evaluation finds that: “At the programme design level, major weakness exist in the lack of ESI-related indicators - such as job creation and levels of social protection - and specific institution-building; this hampers EC monitoring of intervention results, and impedes an accurately informed process to facilitate orientation of follow- up, review, evaluation, recommendations, and assistance with policy fine- tuning.” (p.3).

JC 10.6 The Commission developed and used specific mechanisms to ensure