Developmental Evaluation Pilot Activity (DEPA-MERL)

Speeches Shim

Developmental Evaluation Pilot Activity (DEPA-MERL)
GCFSI PHOTO/LINLIN LIANG

The Developmental Evaluation Pilot Activity (DEPA-MERL) will test the effectiveness of a developmental evaluation (DE) approach, as well as accompanying monitoring and evaluation tools and flexible contracting mechanisms, in achieving effective programming for innovative interventions, untested theories of change, and/or implementation in complex contexts.

The Challenge

For complex interventions or new innovations, traditional midterm and end line evaluations can occur too late to aid in programmatic fine tuning; and often help interventions reach their pre-defined outcomes, rather than work on systems change that may require the redefinition of outputs and outcomes.  

The Innovation

Traditional approaches to monitoring and evaluation focus on making existing processes, structures and systems more effective. DE seeks to improve not only program design, but takes into account the entire complex dynamic system in which the program, project or activity is taking place. DE provides an approach to evaluation that is quick, ongoing, and takes an iterative approach to data collection, analysis and feedback that contributes to timely changes throughout the project cycle and allows for system changes as well as changes in targeted outcomes.

The Approach

Developmental Evaluators are “embedded” within the program, project, or activity to contribute to modifications in program design and targeted outcomes throughout implementation. However, Developmental Evaluators remain independent from implementing staff, providing a third-party perspective to support an adaptive management process. DE does not prescribe a single methodological design, tool, or framework – the evaluation approach taken is based on emerging needs. Methods might include network and outcome mapping, contribution analysis, or other approaches based on information needs. DE is a highly flexible approach and is well-suited for projects under flexible contracting mechanisms in which implementation is likely to change in response to emerging conditions on the ground. 

DEPA-MERL Pilots

DEPA-MERL has piloted four DEs to date to test the effectiveness of the approach. Learn about each pilot and see examples of deliverables and learning products at the links below:

Family Care First (2016-2018): Between 2016 and 2018, the DEPA-MERL team worked with the innovative Family Care First initiative in Cambodia, which leveraged a collective impact approach to “advance transformational solutions that considerably reduce the number of children growing up outside of safe, nurturing, family-based care.” Within FCF’s complex system of partnerships and processes, the DE sought  to better understand how partner relations impact implementation in order to optimize the program’s operational and engagement structure. DEPA-MERL worked closely with Save The Children (integrating partner), The Global Alliance for Children (backbone organization), and nearly 50 local and national partner organizations. 

Sustained Uptake (2017-2018): This DE worked with seven teams across the U.S. Global Development Lab to catalyze the next generation of breakthrough innovations and their uptake within and beyond USAID. Over 22 months, the DE provided extensive evaluative and adaptive management support to the Lab, providing them with evidence on effective and efficient models for both internal and external sustained uptake. The DE further improved teams’ capabilities of achieving ecosystem-level outcomes, and provided tools to continue this work moving forward. 

Bureau for Food Security (2018-2019): As USAID’s Bureau for Food Security (BFS) navigated a structural transformation, BFS identified a need to use learning from current practices to inform the Bureau’s future structures and functions. BFS aimed to use DE to better understand how successful the Bureau’s units were in capturing learning and dealing with challenges related to knowledge management and knowledge sharing. The DE was not fully implemented due to external challenges  and ultimately focused on a narrow set of questions and products.  

Center for Digital Development (2020-2021): In early 2020, USAID launched its first-ever Digital Strategy in order to align the Agency’s vision for development assistance with the world’s evolving digital landscape. The Global Development Lab’s Center for Digital Development (CDD) is responsible for leading the strategy implementation. DEPA-MERL launched a new DE in mid-2020 to support up to four of the Strategy’s 15 implementation initiatives by providing insights into how the initiatives work within their teams and with each other as well as timely input for decision making. For the first time at USAID, the Developmental Evaluator is being embedded remotely into the initiatives due to the covid-19 pandemic. 

Additional Resources from DEPA-MERL

  • A Practical Guide for Evaluators and Administrators - Evidence-based, step-by-step guidance and templates for Developmental Evaluators and firms that manage Developmental Evaluations on how to successfully design and implement a DE.

  • A Practical Guide for Funders -Evidence-based, step-by-step guidance and templates  for funders of Developmental Evaluations on how to successfully design and implement a DE.

  • Developmental Evaluation in Practice: Tips, Tools, and Templates - As a departure from many traditional DE resources that focus on theory, this document showcases DEPA-MERL’s practical lessons learned from direct experience to offer guidance for organizations, managers, and evaluators that seek to implement this approach.

  • Developmental Evaluation Case Study Series - This series of case studies gives examples of implemented DEs around the globe, both inside and outside DEPA-MERL. Each case study provides background on the programming, the DE design and implementation approach, the suitability of DE, and lessons learned for implementation. 

    • Boresha Afya (USAID) -  USAID supports the government of Tanzania to strengthen integrated health services through the Boresha Afya program. This DE, conducted between 2017 and 2021, helps USAID generate real-time evidence and support adaptive management practices to improve the quality, efficiency, utilization, and scalability of integrated health services in Tanzania. 

    • Jilinde (Gates Foundation) - Jilinde’s learning laboratory approach integrates DE at multiple levels to help partners scale up and retain a pre-exposure prophylaxis (PrEP) service with its most vulnerable clients. The DE identifies enablers and challenges throughout its 5-year implementation (2017-2022).

    • CMP (U.S. Department of State) - Beginning in 2015, this DE provides ongoing learning for the project, which uses memory to support peacebuilding and reconciliation in post-war Sri Lankan communities. 

    • Jalin (USAID) - Between 2018 and 2022, this DE supports  Indonesia’s Ministry of Health and DAI Global in their efforts to reduce maternal and neonatal mortality from preventable death through co-created, local solutions. 

    • FCF (DEPA-MERL) - From 2016 to 2018, this DE improved coordination between stakeholders from nearly 50 organizations as they implemented a collective impact model to increase the number of children in Cambodia living in safe, nurturing family-based care.

    • Uptake (DEPA-MERL) - In 2017 and 2018, this DE brought together diverse partners to catalyze the next generation of breakthrough innovations to advance USAID’s mission to support partner countries on their journey to self-reliance.

  • Across-Pilot Analysis - This report shares findings and lessons learned from across three of DEPA-MERL’s DE pilots to answer questions related to emergent learnings, implementation barriers/enablers, and value added/lost with DE compared to traditional evaluation.

Quick Facts

  • Tools: Embedded evaluators, emerging needs tailoring, network mapping, outcome mapping, contribution analysis
  • Funding mechanism: Contract (buy-in option for USAID operating units)
  • Partners: Social Impact (prime), Search for Common Ground, The William Davidson Institute at the University of Michigan
  • Period of Performance: 10/01/2015 – 9/30/2024

Lab Contact: Sophia van der Bijl, svanderbijl@usaid.gov