Documenting Evaluation Use

Speeches Shim

FEATURED

Influential Evaluations

This landmark study traces the impact of eight evaluations which were found to be highly influential and examines why that was so. The report draws lessons for future evaluation design.

DOWNLOAD PDF

ALSO SEE

Evaluation Utilization
A guide to fostering utilization, from Chapter 10 in Designing for Results: Integrating Monitoring and Evaluation in Conflict Transformation Programs


The World Bank Group Impact Evaluations: Relevance and Effectiveness
This volume examines the relevance, quality and effectiveness, or influence, of recent World Bank Group Impact Evaluations, finding that while quality was uniformly high, the direct utilization of World Bank Group Impact Evaluation findings for decision-making has been modest to date.

DOWNLOAD PDF

USAID promotes the utilization of evaluation, including through its CDCS Guidance and Project Design Guidance which call for a discussion of what has been learned from past evaluations that is relevant to the design of new USAID programs and projects. USAID also expects that mid-project evaluations will be used to guide the implementation of on going efforts. Sharing evaluations through USAID’s Development Experience Clearinghouse (DEC) also fosters evaluation use.

The value of an evaluation is determined not by its cost, but rather through its use. While this truism is often repeated, it is still hard to find good examples of use that demonstrate what USAID has gained from the evaluations it conducts. A good way to fix that is to start your own Mission journal on evaluation utilization.

Relative to other topics, the extent to which evaluations have been used for decision-making, and more importantly, made a difference in terms of the effectiveness of the development assistance programs and projects for which they were conducted, has not been well-researched. As far back as 1988, USAID was interested in evaluation use and in a study it commissioned that year, entitled Preliminary Study of the Utilization of A.I.D.'s Evaluation Reports, it learned that about half of the sample of USAID staff they interviewed could identify an immediate, practical use or a conceptual learning point that worked its way into Agency thinking. For none of those instances of utilization however were stories presented of the difference that use made. In 2004, the World Bank carried undertook an effort to identify evaluations that had been influential within the Bank, and to document the returns that accrued from conducting them. This volume, Influential Evaluations, documents eight evaluations that paid off in development results. Other organizations are also beginning to document the development impact of their evaluations, including J-PAL, which has a special "Scale-Ups" segment of its impact evaluation website dedicated to tracing the policy and program effects of these types of evaluations, including J-PAL's research on deworming which, surprisingly at the time, showed this simple treatment to be more effective than many education-based changes in improving school participation, as described below.

School-Based Deworming

Research by J-PAL affiliates Michael Kremer and Edward Miguel has shown that school-based deworming is on of the most cost-effective methods for improving school participation. The evidence from their study has helped inform the debate and has contributed to the scale-up of school-based deworming across 4 countries where over 26 million children have been dewormed since 2009.

Location Partner People Reached
Ethiopia Save the Children USA 0.38 million
The Gambia Gambia Ministry of Education 0.27 million
Kenya Government of Kenya Ministry of Health 3.65 million
Bihar, India Government of Bihar 17.04 million
Andhra Pradesh, India Government of Andhra Pradesh 2.06 million
Delhi, India Government of Delhi 2.65 million
  TOTAL 26.05 million
 << Post-Evaluation Action Follow-Up Report Up Meta-Analyses, Evidence Summits and Systematic Reviews >>

ProjectStarter

BETTER PROJECTS THROUGH IMPROVED
MONITORING, EVALUATION AND LEARNING

A toolkit developed and implemented by:
Office of Trade and Regulatory Reform
Bureau of Economic Growth, Education, and Environment
US Agency for International Development (USAID)

For more information, please contact Paul Fekete.