Use of M&E Information and Outputs by Development Partners

Out of the many stakeholders, donors‘ demand for M&E information produced by the government is highest. Major development partners such as the Asian Development Bank (ADB), World Bank (WB), as well as the UN have taken a keen interest especially on the RM, and have used it mainly to align their strategies to those of the Philippines‘.

ADB, for example, conducted several consultations with the Philippine government, through NEDA, to align its Country Partnership Strategy 2011-2016 and its Country Operations Business Plan 2015-2017 with the priorities indicated in both the PDP and RM (ADB, 2011). Meanwhile, WB was involved with a similar initiative for its own newly- released Country Partnership Strategy for the Philippines 2015-2018. WB‘s overall country

39 strategy supports the country‘s goals to promote inclusive growth, poverty reduction and job creation (World Bank, 2014b). The UN has also engaged the government with an alignment process of its United Nations Development Assistance Framework (UNDAF) 2012-2018, going as far as creating their own results matrices, which is completely based on the PDP-RM (UN, 2011).

These initiatives show donors‘ adherence to the principles of the Paris Declaration, in this case, the principle of alignment (OECD, 2005). On the one hand, while these alignment initiatives are commendable, they also illustrate uncoordinated efforts among donors, which is somehow inconsistent with the principle of harmonization. Even so, alignment of donors‘ strategies with the country‘s priorities also bodes well in aligning donor and country M&E processes, thereby lessening duplication of functions and administrative burden for the government.

40 7 Conclusions and Recommendations

The landscape of M&E in the Philippine government continues to evolve, and has been marked by a number of improvements in the last decade. Progress, specifically at the national level, has been greatly influenced by the principles and practices advocated by the international community in this new era of aid reforms – ownership, alignment, harmonization, accountability and managing for results. And, with such strong emphasis on results in the field of development, the Philippine government is now more than ever relying on M&E as an indispensible tool to strengthen their government processes.

In the last four years, the government has set out on a results agenda that aimed to incorporate results into all areas of public sector management, the highlight initiative of which is the introduction of a results framework to the national plan. The Results Matrices was created as an M&E system (although primarily a monitoring mechanism) to the Philippine Development Plan 2011-2016. The design, as well as the implementation of this M&E system, was beset by many challenges since its inception in 2010.

According to Bedi, et al. (2006), to surpass practical challenges, it is helpful to see M&E as having both supply and demand sides. Any initiative seeking to improve an M&E system should, however, focus on the institutional instead of the technical dimensions of M&E, that is, designing and implementing a coherent M&E system (i.e. supply side), and ensuring the use of M&E information and outputs to inform policy and decision making (i.e. demand side).

This study, thus, sought to determine the strengths and weaknesses of the M&E system of the current Philippine Development Plan (PDP), through a diagnosis of both its supply and demand dimensions.

In terms of supply, a diagnosis was made for three dimensions: plan and policy; organization and coordination mechanisms; and, capacity building.

First, it has been found that the M&E system of the PDP lacks a comprehensive M&E plan to guide its design and implementation. Out of the many components of a typical M&E plan, the only element made available, formally written and disseminated by the government, was an indicator measurement framework, which is contained in the RM. The absence of a

41 comprehensive M&E plan did not have much impact on the coordination aspect of the design of the RMs, as this already follows the existing set up of the planning process. Without the M&E plan, however, there was not enough guidance in how data collection and data management, and even budgets (for related M&E activities) should be executed. It was also a missed opportunity to create a dissemination strategy, which would have helped raising the awareness of a wider range of stakeholder about the availability of M&E information. As we have learned in this study, dissemination is a key strategy to creating demand and use for M&E.

Coordination between government agencies during the design stages of the RM is seen to be one of the strengths of the supply side, as it greatly benefited from the existing, quite organized coordination set up of the planning process. Furthermore, NEDA, who has been traditionally the overseer of the crafting of the national plan, was also able to exercise its influence and generate buy-in and support of line ministries for this M&E initiative.

Capacity, is altogether a different story, as it is one of the weakest elements of the supply side of the M&E system, specifically in terms of the government‘s capacity to conduct evaluations. While the current RM process calls for greater exercise in the government‘s monitoring function (where it is admittedly more advanced), evaluation is still seen to be relevant in its overarching results agenda. A national evaluation policy, whose aim is to guide all evaluation activities in the country, was proposed in 2012, but has yet to take effect. A policy would create an enabling environment for setting standards and harmonizing all M&E initiatives within the government, thereby also contributing to strengthening the supply side of M&E.

As emphasized many times in the paper, a national M&E system, its design process, as well as the information it produces may be sound, but if they are not used, they do not hold value. Authors (Mackay, 2007; Kusek and Rist, 2004) have put forward that the ultimate measure of success of an M&E system is its utilization to support government‘s core functions. This is the demand side of M&E.

A diagnosis was then undertaken for five demand side dimensions: outputs and dissemination; M&E linkage to the budget process; use of M&E information and outputs by Parliament; use of M&E information and outputs by civil society; and, use of M&E information and outputs by development partners.

42 First, the outputs and dissemination dimension of the M&E system is found to be relatively weak. The main output of the M&E process is the Socioeconomic Report, which provides a comprehensive status of the progress of the plan strategies and achievement of indicator targets. NEDA produces the SER in one standard format, as has been traditionally done. NEDA also disseminates the SER to all those who have been involved in the planning process (including the general public by posting it on its website). This kind of dissemination strategy clearly does not encourage the use of M&E information, as evidenced by the limited usage in the part of Parliament and civil society. Here, we see how a weakness in the supply side could cause weak demand, in that stakeholders do not seem to see the value of obtaining M&E information and use them for their own functions. Mackay (2007) noted this situation as a chicken-and-egg problem: there is lack of demand for M&E because of a lack of understanding of what M&E can provide; understanding is limited because of a lack of experience in M&E; and lack of experience in M&E is caused by weak demand.

Even so, if the government, particularly its central M&E unit is truly committed to pursuing its results agenda, it will use its position in the supply side of M&E to ensure that whatever M&E information and outputs it provides is presented in a clear, understandable manner. By presenting better outputs and making them accessible especially to stakeholders with significant interests in M&E information, the government can increase demand and use of M&E to support functions such as budgeting and policy making.

Meanwhile, the ‗linkage to the budget process‘ is the most noteworthy dimension in the demand side. The government‘s performance-informed budgeting (PIB) initiative illustrates how even one working element in the supply side (i.e. well-organized coordination structure between the planning and budget offices) holds the potential to create a space for greater utilization of M&E information. Whether the PIB system will be effective remains to be seen, but if implemented right, the PIB will be a good illustration of how M&E, when essentially made use of, can improve government systems and the way policy making is done.

In its current state, the M&E system of the Philippine Development Plan, certainly has both own strengths and weaknesses. If improvements are to be made, the government must examine in an in-depth manner both the supply and demand sides of.

43 A comprehensive M&E plan should be formulated before the next M&E system of the national planning is rolled out and implemented. If it is also not too late for the existing M&E system, then an M&E plan should also be crafted covering the remaining two years (2015-2016). What can be highlighted is a strategy to conduct program reviews, and if possible, evaluation activities. These will especially be important to learn whether (selected) strategies, expected to have produced results by 2016, were able to deliver.

An M&E plan will also address the challenges related to outputs and dissemination, by including a set of guidelines on how to best present outputs and make them available to the widest possible audience, thereby contributing to the creation of demand. And, if the M&E plan is really to be done comprehensively, it can also look into creating mechanisms for more meaningful participation of all relevant actors.

But perhaps, before crafting an M&E plan, it is proposed that the government engage in a more comprehensive diagnosis or stocktaking of the current state of M&E, including the systems of sector agencies and other non-government actors such as Parliament and civil society. This could be led by NEDA, but as this is such a huge undertaking and resources (manpower and budget) are definitely a constraint, it could be done jointly with willing development partners. Through this diagnosis, the government can explore different dimensions of both supply and demand sides and better understand the interaction between the two, and can therefore prepare a more informed and substantial M&E plan.

If this exercise will not be able to benefit the M&E system of the current plan, whatever can be gleaned from it will definitely be useful to improve the national planning and M&E processes for the next administration.

Lastly, as evaluation is expressed to be high on the government‘s results agenda, it is proposed that the government finalize its national evaluation policy. If decreed, a national evaluation policy is expected to address a number of overarching issues in M&E discussed in this study –budget, capacity-building, mechanisms for participation and utilization of M&E. On its own, of course, a policy does not guarantee a successful M&E system. The government is expected to play a crucial role in steering the policy and the proposals therein towards an efficient and effective implementation.

44 Bibliography

Asian Development Bank (2011) Country Partnership Strategy, Manila, ADB.

Bedi, T., et al (2006) Beyond the Numbers: Understanding the Institutions for Monitoring Poverty Reduction Strategies, Washington, DC, The World Bank.

Caucus of Development NGOs Networks (2011) Assessment of the 2004-2010 Medium Term

Philippine Development Plan, Quezon City, CODE-NGO.

Caucus of Development NGOs Networks (2011) Some Gains, More Frustration: CSOs’

Participation in the Philippine Development Planning (PDP) Process, http://code- ngo.org/home/component/content/article/43-front/237-some-gains-more-frustration-csos-

participation-in-the-philippine-development-planning-pdp-process.html (last consulted: 27

August 2014).

Commission on Audit (2013) 2012 Consolidated Audit Report on Official Development Assistance Programs and Projects, Quezon City: COA.

Congressional Policy and Budget Department (2014) Brief History,

http://www.cpbo.gov.ph/index.php/2012-06-30-13-05-43(last consulted: 27 August 2014). Dale, R. (2004) Evaluating Development Programmes and Projects, 2nd Edition, New Delhi, Sage Publications.

Department of Budget and Management (2014) Mandate,

http://www.dbm.gov.ph/?page_id=343 (last consulted: 20 August 2014).

Department of Budget and Management (2014) Moving from Outputs to Outcomes: The Continuing Evolution of PIB in the Philippines, Manila: DBM.

Edmunds, R. and Marchant, T. (2008) Official Statistics and Monitoring and Evaluation Systems in Developing Countries: Friends or Foes?, Paris, PARIS21.

Executive Order No. 80 (2012) Malacanang, Manila.

Holvoet, N. and Inberg, L. (2012) ―Sector Monitoring and Evaluation Systems in the Context of Changing Aid Modalities: The Case of Uganda‘s Health Sector‖, Study for DGD and BTC in the context of the O*Platform Aid Effectiveness, Antwerp, Institute of Development Policy and Management.

Holvoet, N. and Renard, R. (2006) ―Monitoring and evaluation under the PRSP: Solid rock or quicksand‖, Evaluation and Program Planning, 30 (2007): 66-81.

Holvoet, N. and Rombouts, H. (2008) ―The Challenges of Monitoring and Evaluation Under the New Aid Modalities: Experiences from Rwanda‖, Journal of Modern African Studies, 46 (4): 577-602.

Holvoet, N., Gidmelyn, M. and Inberg, L. (2012) ―Taking Stock of Monitoring and Evaluation Arrangements in the Context of Poverty Reduction Strategy Papers: Evidence from 20 Aid-Dependent Countries in Sub-Saharan Africa‖, Development Policy Review

45 International Monetary Fund (2014) Factsheet Poverty Reduction Strategy Papers,

Washington, DC, International Monetary Fund.

Krause, P. (2012) ―M&E Systems and the Budget‖, in Lopez-Acevedo, G., Krause, P. and Mackay, K. (eds.), Building Better Policies: The Nuts and Bolts of Monitoring and Evaluation Systems, Washington DC, World Bank, 75-86.

Kusek, J. and Rist, R. (2004) A Handbook for Development Practitioners: Ten Steps to a Results-Based Monitoring and Evaluation System, Washington DC, World Bank.

Mackay, K. (2007) How to Build M&E Systems to Support Better Government, Washington, DC, The World Bank.

Memorandum Circular No. 3 (2010) Malacanang, Manila.

National Economic and Development Authority (2011) Philippine Development Plan 2011- 2016 Results Matrices, Pasig, NEDA.

National Economic and Development Authority (2011) ―M&E in the Philippines: Challenges and Prospects‖, First M&E Network Forum, Conference Paper, Pasig, NEDA.

National Economic and Development Authority (2011) Philippine Development Plan 2011- 2016, Pasig, NEDA.

National Economic and Development Authority (2011) The Monitoring and Evaluation Network Philippines, http://devplan.neda.gov.ph/m&e-network/ (last consulted: 26 August 2014)

National Economic and Development Authority (2012) CY 2011 ODA Portfolio Review Report. Pasig, NEDA.

National Economic and Development Authority (2013) ―Tightening the Government‘s Results Framework‖ Workshop on Results Based Monitoring and Evaluation, Workshop Presentation, Pasig, NEDA.

National Economic and Development Authority (2013) ―Draft National Evaluation Policy‖ National Economic and Development Authority (2013) CY 2012 ODA Portfolio Review Report. Pasig, NEDA.

National Economic and Development Authority (2014) Philippine Development Plan 2011- 2016 Revalidated Results Matrices, Pasig, NEDA.

OECD (2005) Paris Declaration on Aid Effectiveness. Paris, OECD/DAC

OECD (2008) 2008 Survey on Monitoring the Paris Declaration: Making Aid More Effective By 2010. Paris, OECD/ DAC.

OECD (2008) Emerging Good Practice in Managing for Developmemt Results Source Book

46 OECD (2009) Supporting evaluation capacity development in partner countries – next steps for the Network, Room Document, Paris, OECD/DAC.

OECD (2011) Aid Effectiveness 2011: Progress in Implementing the Paris Declaration – Volume II Country Chapters Philippines. Paris, OECD/DAC.

OECD-DAC (2002) Glossary of Key Terms in Evaluation Results Based Management, Paris, OECD.

Porter, S. and Goldman, I. (2011) ―A Growing Demand for Monitoring and Evaluation in Africa‖, African Evaluation Journal, 1(1): 1-9.

Republic Act No. 8182: Official Development Assistance Act of 1996 (1996) Congress of the Philippines, Metro Manila.

Robinson, M. and Last, D. (2009) A Basic Model of Performance-Based Budgeting,

Washington, DC, International Monetary Fund.

Segone, M. (2010) ―Moving from Policies to Results by Developing National Capacities for Country-Led Monitoring and Evaluation Systems‖, in Segone, M. (ed), From policies to results: developing capacities for country monitoring and evaluation systems, New York, UNICEF, 22-43.

Stern, E. (2008) Thematic Study on the Paris Declaration, Aid Effectiveness and Development Effectiveness, Koege, Ministry of Foreign Affairs of Denmark.

The Global Fund (2011) Monitoring and Evaluation Toolkit: HIV, Tuberculosis, Malaria and Health and Community Systems Strengthening, Geneva, The Global Fund.

Tuaño, P. (2011) ―Philippine Non-Government Organizations (NGOs): Contributions, Capacities, Challenges‖, in Jose, L. (ed.) Civil Society Organizations in the Philippines, A Mapping and Strategic Assessment, Quezon City, Civil Society Research Institute, 9-46. United Nations (2012) United Nations Development Assistance Framework 2012-2018,

Makati City, United Nations in the Philippines.

Valadez J. and Bamberger, M. (1994) Monitoring and Evaluating Social Programs in Developing Countries: A Handbook for Policymakers, Managers and Researchers,

Washington, DC, The World Bank.

World Bank (2007) Results-Based National Development Strategies: Assessment and Challenges Ahead, Washington, DC, World Bank.

World Bank (2014) Comprehensive Development Framework,

http://web.worldbank.org/WBSITE/EXTERNAL/PROJECTS/STRATEGIES/CDF/0,,pageP

K:60447~theSitePK:140576,00.html (last consulted: 18 August 2014).

47 Annex 1

Checklist/Guide Questions for Diagnostic Exercise

A. Supply Side

Plan and Policy

● Is there a single PRS monitoring strategy or master plan? What is its status? Is it being implemented?

Organization and Coordination Mechanisms

● Which mechanisms, such as committees or working groups, have been established to facilitate coordination among agencies and stakeholders?

● Is their composition stable?

● Are various stakeholders represented at an appropriate level to reflect and ensure their commitment? ● Is there a functioning secretariat of the PRS monitoring system?

● Are the meetings organized in a way that supports coordination? ● Are the information flows adequate to support coordination? ● Is the burden on participants excessive?

Capacity Building

● Is there an overall capacity-building program or plan? Does it identify needs and gaps? Is it clearly prioritized? Is it costed and funded?

● Are development partners key funders? What are their funding trends? How sustainable and predictable is their funding? Are they supporting the overall system or only selected activities by certain actors? Is the government providing guidance to development partners on supporting capacity development?

● Are development partners funding technical assistance in the design and strengthening of the PRS monitoring system? Are skills being transferred to the country as a result of this assistance?

● Are substantive capacity-building efforts in monitoring, analysis, and evaluation currently under way in the country? Are they directly related to the PRS monitoring system? Are they at the national, sectoral, or project