Sustainable development: Will we ever know if we’ve achieved it?
The UN has just passed a series of Sustainable Development Goals (SDGs). Seventeen bold goals for the world: No more hunger! Education for all! Watch the video to see what they’ve agreed to work toward. It’s a beautiful picture of what humans could achieve.
Leave it to Dr. Tom Schwandt of the University of Illinois at Urbana-Champaign to bring a sobering voice to the party. How will we ever measure progress toward those goals? And if we can’t measure them, how will we ever know if we achieved them?
“If evaluation is a religion,” he says, referencing an earlier comment by Dr. Deborah Rugg, “we might have to do a little praying.” He brings up five issues the UN will face in trying to measure progress.
1) The SDGs are meant to be universal, but it’s not clear yet what policies and programs will be needed. The challenges to achieving sustainable development are different for low, middle and high income countries. How can we collapse both interventions and their measurement into a global summary, or even a global picture? How can we structure meaningful reporting on a global scale?
2) There is an extremely strong focus on data, both availability and quality. There’s an entire UN subgroup assigned to coming to agreement on a set of indicators for the 17 SDGs. So far they’ve come to agreement on 159 indicators and are still debating another 20 or so. Big data, citizen-generated data, other types of data, what are the risks of data collection and ownership? The real danger here, Schwandt says, is that a focus on data for indicators leads to focus on monitoring which is not the same thing as evaluation. How do we do evaluation without falling into the monitoring trap?
3) Schwandt suggests reading a blog post by Caroline Heider of the Independent Evaluation Group at the World Bank. In sum, she points out the 17 SDGs are so complex, integrated and interrelated that standard linear models for measuring and impact will not work. We will have to develop new strategies for incorporating systems thinking and methods of doing systems oriented evaluation.
4) Attribution and contribution: how can we possibly measure which policies and programs and in what dosage work? And in which settings?
5) Finally, there’s that tricky issue of scale. It’s hard enough to measure some of these outcomes at the country level. How can we roll up findings to the global level, taking into account inter- and intra-regional differences in measurement.