Rigorous and systemic evaluations have the potential to leverage the impact of international organizations well beyond simply their ability to finance programs.
Credible impact evaluations are international public goods: the benefits of knowing that a program works or does not work extend well beyond the organization or the country implementing the program.11 Programs that have been shown to be successful can be adapted for use in other countries and scaled up within countries, while unsuccessful programs can be abandoned. Through promoting, encouraging, and financing rigorous evaluations (such as credible randomized evaluations) of the programs they support, as well as of programs supported by others, the international organizations can provide guidance to the international organizations themselves, as well as other donors,
governments, and NGOs in the ongoing search for successful programs. Moreover, by credibly establishing which programs work and which do not, the international agencies can counteract skepticism about the possibility of spending aid effectively and build long- term support for development. Just as randomized trials for pharmaceuticals
revolutionized medicine in the 20th Century, randomized evaluations have the potential to revolutionize social policy during the 21st.
11 In fact, the benefits of a credible evaluation are often negative for the person or organization promoting the program.
References
Angrist, Joshua, Eric Bettinger, Erik Bloom, Elizabeth King, and Michael Kremer (2002) “Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment,” American Economic Review, 92(5), 1535-58.
Angrist, Joshua and Alan Krueger (1999) “Empirical Strategies in Labor
Economics,” in Handbook of Labor Economics, Vol. 3A. Orley Ashenfelter and David Card (Eds.). Amsterdam: North Holland, 1277-1366.
Angrist, Joshua and Alan Krueger (2001) “Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments,” Journal of Economic Perspectives, 15(4), 69-85.
Angrist, Joshua and Victor Lavy (1999) “Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement,” Quarterly Journal of Economics, 114(2), 533-575.
Ashenfelter, Orley, Colm Harmon, and Hessel Oosterbeek (2000) “A Review of
Estimates of Schooling/Earnings Relationship, with Tests for Publication Bias,” NBER working paper #7457.
Attanasio, Orazio, Costas Meghir and Ana Santiago (2001) “Education Choices in Mexico: Using a Structural Model and a Randomized Experiment to Evaluate PROGRESA,” mimeo, Inter-American Development Bank.
Banerjee, Abhijit, Shawn Cole, Esther Duflo and Leigh Linden (2003) “Improving the Quality of Education in India: Evidence from Three Randomized Experiments,” mimeo, MIT.
Banerjee, Abhijit and Ruimin He (2003) “The World Bank of the Future,” American Economic Review, Papers and Proceedings, 93(2): 39-44.
Banerjee, Abhijit, Suraj Jacob, and Michael Kremer with Jenny Lanjouw
and Peter Lanjouw (2001) “Promoting School Participation in Rural Rajasthan: Results from Some Prospective Trials,” mimeo, MIT.
Banerjee, Abhijit and Michael Kremer with Jenny Lanjouw and Peter Lanjouw (2002) “Teacher-Student Ratios and School Performance in Udaipur, India: A
Prospective Evaluation,” mimeo, Harvard University.
Behrman, Jere, Piyali Sengupta and Petra Todd (2002) “Progressing through PROGESA: An Impact Assessment of a School Subsidy Experiment in Mexico,” mimeo, University of Pennsylvania.
Bertrand, Marianne, Esther Duflo and Sendhil Mullainathan (2002) “How Much Should We Trust Difference in Differences Estimates?” NBER working paper #8841.
Besley, Timothy and Anne Case (2000) “Unnatural Experiments? Estimating the Incidence of Endogenous Policies,’ Economic Journal, 110(467), F672-F694.
Bobonis, Gustavo, Edward Miguel and Charu Sharma (2002) “Iron Supplementation and Early Childhood Development: A Randomized Evaluation in India,” mimeo, University of California, Berkeley.
Buddlemeyer, Hielke and Emmanuel Skofias (2003) “An Evaluation on the Performance of Regression Discontinuity Design on PROGRESA,” Institute for Study of Labor, Discussion Paper No. 827.
Campbell, Donald T. (1969) “Reforms as Experiments,” American Psychologist, 24, 407-429.
Card, David (1999) “The Causal Effect of Education on Earnings,” in Handbook of Labor Economics, Vol. 3A. Orley Ashenfelter and David Card (Eds.). Amsterdam: North Holland, pp. 1801-63.
Case, Anne (2001) “The Primacy of Education,” mimeo, Princeton University.
Chattopadhyay, Raghabendra and Esther Duflo (2001) “Women as Policy Makers: Evidence from a India-Wide Randomized Policy Experiment,” NBER working paper # 8615.
Cook, Thomas D. (2001) “Reappraising the Arguments Against Randomized
Experiments in Education: An Analysis of the Culture of Evaluation in American Schools of Education,” mimeo, Northwestern University.
Cronbach, L. (1982) Designing Evaluations of Educational and Social Programs. San Francisco: Jossey-Bass.
Cronbach, L., S. Ambron, S. Dornbusch, R. Hess, R. Hornik, C. Phillips, D. Walker and S. Weiner (1980) Toward Reform of Program Evaluation. San Francisco: Jossey- Bass.
Cullen, Julie Berry, Brian Jacob and Steven Levitt (2002) “Does School Choice Attract Students to Urban Public Schools? Evidence from over 1,000 Randomized Lotteries,” mimeo, University of Michigan.
DeLong, J. Bradford and Kevin Lang (1992) “Are All Economic Hypotheses False? ”
Journal of Political Economy, 100(6) (December), 1257-72.
in Development Economics Conference Proceedings, The World Bank.
Duflo, Esther (2001) “Schooling and Labor Market Consequences of School Construction in Indonesia: Evidence from an Unusual Policy Experiment,”
American Economic Review, 91(4): 795-814.
Duflo, Esther and Emmanuel Saez (forthcoming) “The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment,” Quarterly Journal of Economics.
Gertler, Paul J., and Simone Boyce (2001) “An Experiment in Incentive-Based Welfare: The Impact of PROGRESA on Health in Mexico,” mimeo, University of
California, Berkeley.
Glazerman, Steven, Dan Levy, and David Meyers (2002) “Nonexperimental Replications of Social Experiments: A Systematic Review.” Mathematica Policy Research, Inc. Interim Report/Discussion Paper.
Glewwe, Paul, Nauman Ilias, and Michael Kremer (2003) “Teacher Incentives,” mimeo, Harvard University.
Glewwe, Paul, Michael Kremer, Sylvie Moulin, and Eric Zitzewitz (2003) “Retrospective vs. Prospective Analyses of School Inputs: The Case of Flip Charts in Kenya,” forthcoming, Journal of Development Economics.
Glewwe, Paul, Michael Kremer, and Sylvie Moulin (2002) “Textbooks and Test Scores: Evidence from a Prospective Evaluation in Kenya,” mimeo, Harvard University.
Hanushek, Eric A. (1995) “Interpreting Recent Research on Schooling in Developing Countries,” World Bank Research Observer, 10 (August), 227-246.
Heckman, James, Hidehiko Ichimura, Jeffrey Smith, and Petra Todd (1998)
“Characterizing Selection Bias Using Experimental Data,” Econometrica, 66(5), 1017-98.
Heckman, James, Hidehiko Ichimura, and Petra Todd (1997) “Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Program,” Review of Economic Studies, 64(4), 605-54.
Heckman, James, Robert Lalonde, and Jeffrey Smith (1999) “The Economics and Econometrics of Active Labor Market Programs,” in Handbook of Labor Economics: Volume 3. O. Ashenfelter and D. Card, eds. North Holland.
Heckman, James, Lance Lochner, and Christopher Taber (1998) “General Equilibrium Treatment Effects: A Study of Tuition Policy,” NBER working paper #6426.
Imbens, Guido and Joshua Angrist (1994) “Identification and Estimation of Local Average Treatment Effects,” Econometrica, 62(2), 467-475.
Kremer, Michael (2003) “Randomized Evaluations of Educational Programs in
Developing Countries: Some Lessons,” American Economic Review Papers and Proceedings, 93(2): 102-115.
Kremer, Michael, Sylvie Moulin, and Robert Namunyu (2002) “Unbalanced Decentralization,” mimeo, Harvard University.
Krueger, Alan (1999) “Experimental Estimates of Education Production Functions,”
Quarterly Journal of Economics, 114(2), 497-532.
LaLonde, Robert (1986) “Evaluating the Econometric Evaluations of Training with Experimental Data,” The American Economic Review, 76(4), 604-620.
Meyer, Bruce D. (1995) “Natural and quasi-experiments in economics,” Journal of Business and Economic Statistics, 13(2), 151-161.
Miguel, Edward and Michael Kremer (2003a) “Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities,” forthcoming in
Econometrica.
Miguel, Edward and Michael Kremer (2003b) “Social Networks and Learning About Health in Kenya,” mimeo, Harvard University.
Morduch, Jonathan (1998) “Does Microfinance Really Help the Poor? New Evidence from Flagship Programs in Bangladesh,” mimeo, Princeton University.
Narayanan, Deepa, (ed), Empowerment and Poverty Reduction: A Sourcebook, The World Bank, 2000.
Pandey, Raghaw Sharan (2000) Going to Scale With Education Reform: India’s District Primary Education Program, 1995-99. Education Reform and Management Publication Series, Volume I, No. 4. (Washington: World Bank, 2000).
Pitt, Mark and Shahidur Khandker (1998) “The Impact of Group-Based Credit Programs on Poor Households in Bangladesh: Does the Gender of Participants Matter?”
Journal of Political Economy, 106(5), pp. 958-996.
Pritchett, Lant (2003) “It Pays to be Ignorant,” forthcoming, Journal of Policy Reform.
Rosenbaum, Paul R. (1995) “Observational Studies,” In Series in Statistics. New York; Heidelberg; London: Springer.
Shultz, T. Paul (2003) “School Subsidies for the Poor: Evaluating the Mexican PROGRESA Poverty Program,” forthcoming, Journal of Development Economics.
Vermeersch, Christel (2002) “School Meals, Educational Achievement, and School Competition: Evidence from a Randomized Experiment,” mimeo, Harvard University.