The OECD DAC just published their long-awaited guidance on the appropriate application of the (in)famous DAC evaluation criteria which were updated and adopted on December 2019. Everyone who uses the criteria should study this useful guidance in depth, as this thoughtful post by Michaela Raab also emphasises. Megan and those who supported her deserve credit for what they have done. The guidance will be critical in helping to avoid the pitfalls in using the criteria, which many said were the main problem with the original set.
The guidance is an important step. The updated criteria have even more easy-to-ignore demands stacked under each one. For more about the debates around their merit and usefulness, see my critical 2018 blog post series on the DAC criteria. The posts solicited many thoughtful comments from around the world. But despite this helpful guidance I am still not a fan. There is now more recognition of the need to demand evaluations informed by a complex adaptive systems view of the world, but the criteria are still too safe and comfortable amidst a world in turmoil and in dire need of systems change and transformative action.
Selecting and defining evaluation criteria are a serious professional matter. Together with evaluation questions, criteria lay out the contours for an assessment. The worldview, assumptions and values that underlie the criteria we use shape the findings, judgments, recommendations and lessons that flow from each assessment.
This is of particular importance in the case of these DAC criteria, as they are set to remain enormously powerful, especially in the Global South, despite expressions of modesty from persons involved in their design and promotion. This is why they deserve in-depth, ongoing scrutiny by well-informed, thoughtful persons who have NOT been steeped in outdated narratives about how the world – and development – are supposed to work.
For important additional or alternative perspectives, see the approaches of ALNAP, the Network of Southern Think-tanks Africa (NeST Africa) with its contributions to South-South cooperation with its very different underlying principles and hence implied criteria, and Michael Quinn Patton in this article, which reflects my view that the criteria are fine for ‘business as usual summative and accountability evaluations’, but remain inadequate for addressing systems change and transformation from local to global level.
Yet that is what we need to do in the aftermath of the COVID-19 pandemic and the looming crises resulting from the Anthropocene and extreme capitalism. The DAC criteria still do not encourage us well enough to play a more dynamic and thoughtful role in facing the world’s most important and most urgent challenges.
So, it is essential that commissioners do not fall into new comfort zones, but look afresh at each evaluation to see which criteria out of a broad range will add enough value and be bold enough to get to the essence of what we need to understand and assess to ensure that evaluation can make a difference beyond some incremental inputs. Otherwise we will continue to fiddle at the edges of “development”, whether in the Global South, or in the Global North where all well-off societies need to rethink their relationship with the whole community of life on which we all depend.