Reconstruction of the intervention logic

What were the rationale and objectives of the NFP as formulated in 2002 and reformulated in 2008, and are these still valid?

The overall aim of the NFP is to help alleviate qualitative and quantitative shortages of skilled manpower, within the framework of sustainable capacity-building directed towards reducing poverty in developing countries. In the tender documents of the Ministry the intervention logic is described along broad lines, as follows:

“The aim of the NFP, as mentioned above, is ‘to help alleviate quantitative and qualitative shortages of skilled manpower at mid-career level, in the context of capacity building within the framework of poverty reduction in developing countries’. NFP specifically aims to address the needs for training in the short-term,

42 Evaluation of the Netherlands Fellowship Programme (NFP) 2002 - 2010

focusing on capacity building in a broad spectrum of governmental, private and non-governmental organizations (educational institutes, planning institutes, ministries, private companies, etc.) (Policy framework 2001). The need for training has to be evident within the context of an organization. This implicates that applicants have to be nominated by their employers. By addressing these shortages, the Programme is expected to contribute to qualitatively better functioning organizations, companies, etc. and thereby aims to give a qualitative impulse to society.”

In the Memorandum of assessment (Beoordelingsmemorandum) the intervention logic is briefly formulated as follows:

 Mid-career professionals become better educated by following an education or course (Master, Short Course, PhD or refresher course) at a Dutch education institute;

… therefore the employers involved (government, private and non-governmental organisations and business) in developing countries get better educated staff;

… therefore organisations will improve their functioning.

 50% of the fellows will be women, and people from deprived groups and regions will have more chances to participate in the programme.

This intervention logic was put into a result oriented NFP Framework for outcomes, result areas and result indicators at policy level as well as on programme level, which was formally agreed between the Ministry and Nuffic (see Table 2.3 and Table 2.4).

Table 2.2 Policy level: Overview of goals, outcomes and performance indicators

Goals Outcomes Result indicators

Capacity building at individual level Fellow develops more and better applicable knowledge, skills and attitude.

 Number of fellows that completed their NFP education or training;

 More promotion possibilities within the organisation. Capacity building at organisational

level

Fellow is capable to apply the learned knowledge and skills within the organisation.

The organisation where the alumni work improve their functioning, products and services as a result of the NFP education and training of their employees.

50% of the grants go to women At least 50% of the NFP fellows is female.

Percentage and number of women that completed an NFP education or training.

50% of the NFP budget is granted to candidates from Sub-Saharan Africa

At least 50% of the NFP budget is spent on fellows from Sub-Saharan Africa.

Percentage and number of mid- career professionals from Sub- Saharan Africa that completed an NFP education or training. More NFP grants are made

available for disadvantaged groups and areas

Share of grants for disadvantaged groups and areas has considerably increased. For each country disadvantaged groups and areas are defined in advance.

Increase in number of mid-career professionals from disadvantaged groups or areas that completed their NFP education or training.

Courses/modules performed in the NFP countries

Demonstrable active involvement of local experts in the education and training.

Increase in courses/modules regionally performed:

 Number of MA courses with a regional performed module;

 Number of short courses with a regional performed module.

44 Evaluation of the Netherlands Fellowship Programme (NFP) 2002 - 2010

Table 2.3 Programme level: overview of performance based agreement between DCO/ministry and Nuffic

Goal Performance criteria Performance indicator

Efficiency Number of programme outputs that has been agreed in the annual plan/reports.

 Number of applications and granted fellowships, broken down in various back ground characteristics;

 Number of produced programme descriptions, guidelines, documentation, information systems etc., composition and changes in het NFP course list, selection procedures;

 Number of implemented monitoring and evaluation activities.

Efficiency Exhaustion of programme budgets in relation to achieved results.

Budget versus spending regarding implementation and management within the agreed policy

framework. Advices embassies Embassies demonstrably make

use of their knowledge and insight to advice on the applicants for the NFP.

Timely communication on priority policies. Active involvement in selection procedure based on concrete advices (only in the new selection procedure SOL). Adaptive power /responsiveness Capability of the NFP management

to change management and implementation periodically.

Changes and improvements as a result of internal and external M&E activities.

Based on these documents and interviews with the ministry and Nuffic we formulated a

comprehensive intervention logic to ensure that all stakeholders have a common understanding of what it is meant to achieve and how this is to be done.

Figure 2.5 Intervention logic NFP

The building blocks of the intervention logic as detailed in Figure 2.5 contain a series of underlying assumptions relating to the mechanisms according to which the NFP is assumed to achieve its outputs and results. If an activity is started in the sense of one of modalities in the programme (Master, Short Course, PhD, Tailor Made Training or Refresher Course), and if certain assumptions are met, then it can be expected that the programme will reach its expected output in the form of:21

 Number of graduates /degrees/certificates (PhD, MA, short courses);  Number of participants in refresher courses.

Moreover, it is assumed that these outputs, under certain conditions will reach the expected outcomes. At the individual level the (main) expected outcomes are:

 Improved knowledge and skills, including training skills;

 Involvement in professional networks with fellows/colleagues abroad, i.e. in Netherlands;  Improved job performance/working practice;

 Improved career and employability prospects.

21 _{The outputs and outcomes in italics were specifically mentioned in an internal policy note by the ministry of Foreign affairs}

46 Evaluation of the Netherlands Fellowship Programme (NFP) 2002 - 2010 At the level of the organisation the (main) expected outcomes:  Improved quality of staff/ Reduced shortages of skilled manpower;  Improved quality of services and products;

 Improved functioning of the organisation.

At the national level of a country the (main) expected outcomes (and impacts) are:  Improved quantitative and qualitative skilled manpower;

 Reduced shortages of skilled manpower;

 Improved economic and social development, i.e. for deprived regions and target groups;  Poverty alleviation (longer term impact).

Although not explicitly mentioned in the programme documentation it was frequently mentioned in the interviews with various stakeholders that the NFP should also be seen as a political instrument in building and retaining good relationships with strategic partner countries.

Remarks concerning the intervention logic

From a general perspective the intervention logic is reasonable, focusing on the individual employees developing new skills and knowledge, eventually leading to better quality products and services. The linkages between the activities and outcomes and impacts depend on a number of assumptions which are essential to the chain of relations.

At the level of the individual candidate it is essential that good quality candidates are selected, that do have the right base of knowledge and competences to successfully follow an NFP education or training, which will return to their organisation after they have finished their education or training. To improve their functioning they should be given possibilities to apply their newly learned skills and knowledge.

For the organisation to improve it’s functioning it is necessary that candidates are selected that are at least likely to have an impact on the quality of goods and services of the organisation, allowing the fellow to apply and transfer the skills and knowledge (s)he acquired during the NFP education or training. The organisation must show ‘openness to new knowledge and skills’, and must be willing to support the utilisation of new skills and knowledge in daily work practice; merely fellowships are not sufficient to improve the functioning of organisations. Moreover, logically NFP fellows are more likely to have an impact on the organisation if there are more of them, relative to the size of the organisation.

Of course, It is self-evident that to improve skills and competences at the individual level and to improve the delivery of products and services at the level of the organisation the education and training provided under the NFP must be of good quality.

Since the general objective of the NFP is set at the level of organisations, impacts beyond the level of the organization are largely beyond outside the scope of this evaluation.

In the intervention logic obvious differences between the NFP modalities (e.g. short courses versus Master Courses) are mostly neglected, whereas it can be assumed that the underlying assumptions regarding their effectiveness might be different.