Introduction to the role of the OECD/DAC criteria in impact evaluation

In today's global landscape, the effective evaluation of development initiatives remains a critical challenge. The OECD DAC for international development cooperation serves as a cornerstone, providing essential standards and guidelines for measuring, assessing, and reporting official development assistance. In impact evaluation specifically, the OECD DAC's frameworks and methodologies offer invaluable tools for evaluating the outcomes and effectiveness of development interventions, enabling stakeholders to make evidence-based decisions, optimize resource allocation, and ultimately drive meaningful and sustainable impact in the pursuit of global development goals. However, despite its significance, leveraging the OECD/DAC criteria optimally poses several challenges, ranging from adapting evaluation methods to complex humanitarian contexts to ensuring accountability and transparency throughout the process.

You can read this article in French and Spanish too.

Understanding the origins of the OECD/DAC criteria and importance

The journey of utilizing the OECD/DAC criteria began in 1991 when the Development Assistance Committee established overarching principles to guide evaluation processes. These principles, later refined into criteria, have since become a standard in assessing development initiatives. The widespread adoption of these criteria simplifies evaluation synthesis, identifies common weaknesses in humanitarian actions, and fosters global collaboration among evaluators.

These six OECD/DAC criteria encompass the following questions:

  • Criterion 1: Effectiveness, is the intervention achieving its objectives?
  • Criterion 2: Impact, what difference does the intervention make?
  • Criterion 3: Efficiency, how well are resources being used?
  • Criterion 4: Relevance, is the intervention doing the right thing?
  • Criterion 5: Coherence, how well does the intervention fit?
  • Criterion 6: Sustainability, will the benefits of the intervention last?

Harnessing the power of impact evaluation

The core objective of employing the OECD/DAC criteria is to conduct impact evaluations systematically. Impact evaluation involves assessing the direct outcomes of interventions, providing empirical evidence on their effectiveness. It not only guides decision-making processes but also facilitates learning and accountability, enabling organizations to refine their strategies and better support affected communities.

A systematic and impartial examination of humanitarian action is intended to draw lessons to improve policy and practice, enhance accountability and has the following characteristics:

  • It is commissioned by or in cooperation with the organization(s) whose performance is being evaluated.
  • It is undertaken either by a team of non-employees (external) or by a mixed team of non-employees (external) and employees (internal) from the commissioning organization and/or the organization being evaluated.
  • It assesses policy and/or practice against recognized criteria (e.g., the DAC criteria).
  • It articulates findings, draws conclusions, and makes recommendations—things that an organization can take and turn into an action plan.
  • It goes beyond describing or measuring impacts that have occurred to seeking to understand the role of the intervention in producing these (causal attribution).
  • It uses a variety of methods to identify causal attribution and examine unintended impacts.

In humanitarian contexts, evaluating impact poses unique challenges. From accessing key informants in volatile environments to navigating ethical considerations and power dynamics, humanitarian evaluations require careful planning and adaptation. The urgency of response further necessitates swift evaluation to inform timely decision-making.

The context plays a crucial role, when implementing an evaluation. Humanitarian evaluations are often undertaken during periods of severe disruption, which, in the case of complex emergencies, can be prolonged, thus:

  • Accessing key informants can be a challenge. People are moving and changing locations, which makes it a challenge to locate them.
  • The rapid project set-up results in the establishment of MEAL systems that are only partially implemented. It is crucial to have a MEAL system in place, as the monitoring will then feed into the evaluation.
  • There is a growing significance of accountability to affected populations.
  • When functioning in a context that is quite fragile and there are specific power dynamics in specific regions between governments and different organizations, credibility can be impacted.
  • Following humanitarian principles and ethical considerations could impact the design of impact evaluations and their overall implementation.
  • In complex and ever-changing contexts, it is essential to make flexible adjustments, such as reducing data collection and shortening surveys. We must carefully consider the methods used in conducting evaluations.
  • The urgency necessitates prompt evaluation to swiftly inform decision-making processes.

Principles guiding impact evaluation

To effectively navigate these challenges, impact evaluation must adhere to key principles:

  • Relevance and Appropriateness: The assessment should be customized to fit the specific context and the requirements of the affected population, keeping in mind the characteristics of the humanitarian crisis and the operational setting.

  • Rigour and Credibility: Using methodologies that produce dependable and credible outcomes, ensuring trust among all stakeholders.

  • Feasibility: Performing evaluations that are feasible and realistic given the available resources and on-the-ground circumstances.

  • Ethics and Equity: Guaranteeing that the evaluation process upholds the dignity and rights of all participants, especially vulnerable groups, and encompasses diverse viewpoints.

  • Transparency and Participation: Involving stakeholders at every stage of the evaluation process and being transparent.

  • Usefulness and Learning: Providing practical recommendations that enhance humanitarian practices and policies.

  • Accountability: Requiring humanitarian actors to answer to funders and the impacted population for their actions.

Conclusion

In conclusion, leveraging the OECD/DAC criteria for impact evaluation presents a powerful opportunity to enhance the effectiveness and accountability of development assistance. The use of the OECD/DAC criteria is crucial for improving the evaluation of development projects worldwide. These criteria provide vital standards and guidelines that promote transparency, accountability, and effectiveness in achieving global development objectives. Despite challenges, such as navigating complex humanitarian contexts and ensuring accountability, the adoption of these criteria has become standard practice since 1991. Adhering to key principles such as relevance, rigor, and transparency is essential for navigating the complexities of humanitarian evaluation. Ultimately, the ongoing journey of enhancing impact evaluation with the OECD/DAC criteria holds the potential to drive meaningful and sustainable global development outcomes.

Resources and further reading: