Artificial Intelligence (AI) has an array of potential as an effective tool that can assist the development work. For one, AI can be a transformative force to accelerate sustainable development goal (SDG) targets such as climate change mitigation, healthcare, education, and economic growth. Through the lens of the public sector, AI can help enhance efficiency and productivity while improving decision and policy-making.

The public sector can learn from the experience of international organizations on how they have used AI to maximize efficiency and minimize risks, which, in the long run, can help boost productivity.

Marc Segone, Evaluation Officer Director of the United Nations Population Fund (UNFPA), in his presentation during the 2023 Asian Evaluation Week held in Thailand last 11-14 September, talked about the various stages in the evaluation process where AI can be applied, including data collection, analysis, interpretation, communication, synthesis, and report generation. The UNFPA also recognizes the importance of enhancing digital and AI literacy to bridge the digital divide, which can also be applied in the public sector.

Uyen Kim Huynh, Innovation Specialist at the Evaluation Office of the United Nations Children’s Fund (UNICEF), and Martin Prowse, Evaluation Specialist of Green Climate Fund (GCF), also shared their insights on how AI has helped them improve their efficiency.

At UNICEF, AI is tapped in the organization’s evaluation processes, particularly through the use of Natural Language Processing (NLP), for a better understanding of the holistic impact of UNICEF’s interventions, especially in multidimensional goals. These include analyzing priorities of interventions, scoping and synthesizing evaluations, generating baselines, and examining social data from sources like X (Twitter). AI tools also assist UNICEF in processing and cleaning large volumes of data efficiently, making it ready for analysis. AI further helps integrate diverse data sets from various sources, which is crucial for constructing baselines and counterfactual values. This can also be applied in public sector organizations that help decision-makers develop informed, evidence-based policies and recommendations.

While GCF has not been using generative AI yet, it explores the tool’s potential utilization in the future, especially in addressing the challenge of measuring adaptation in climate interventions. One significant challenge in climate evaluation is measuring adaptation, as it has no concrete metric similar to mitigation’s carbon footprint. AI may be helpful in synthesizing existing Climate Change Adaptation’s (CCA) monitoring and evaluation frameworks and other CCA activities, such as ensuring transparency, longitudinal data usage, and its alignment with the national contexts.

Still, AI also has its known harms and potential risks. These include human-AI conflict (difference in interpretation between the two), loss of control and technological dependency, unemployment fears and socio-economic disparities, lack of legal and regulatory frameworks, and ethical concerns.

Strong ethical guidelines and international regulations are crucial to reduce potential harm and mitigate AI-related risks. Implementing activities such as robust testing, data quality control, transparency, regular monitoring, and human oversight are also essential. Finally, automation must be balanced with human expertise and oversight in AI-powered evaluation.

Watch the session recording here.

Key Takeaways:

  • AI holds promise in improving planning, decision intelligence, predictive analytics, and other systems that support evaluators in drawing more precise conclusions and making recommendations. As AI technology matures, its impact on evaluation practices is expected to grow. However, there is a need for regulation in using generative AI, ensuring data privacy, sensitivity, and responsible usage within organizations.
  • The evaluation community can be pivotal in ensuring responsible and ethical AI power evaluation. The transdisciplinary nature of evaluation can contribute to assessing AI technologies’ merit and impact while identifying biases and unfair outcomes. Collaboration with internal stakeholders and external partners and developing a clear vision and leadership are essential for successful AI integration into the evaluation function.