AI for GovTech

Exploring the use of LLMs for GovTech Benchmark Operationalization

Master Thesis (2024)
Author(s)

C. Snoeij (TU Delft - Technology, Policy and Management)

Contributor(s)

N Nitesh – Mentor (TU Delft - Information and Communication Technology)

J.M. Durán – Graduation committee member (TU Delft - Ethics & Philosophy of Technology)

Faculty
Technology, Policy and Management
More Info
expand_more
Publication Year
2024
Language
English
Graduation Date
28-06-2024
Awarding Institution
Delft University of Technology
Programme
['Engineering and Policy Analysis']
Faculty
Technology, Policy and Management
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

This research explores the use of Artificial Intelligence (AI), specifically Large Language Models (LLMs), into the operationalization of Government Technology (GovTech) benchmarks to increase their utility for policymakers. Research and practice consistently highlight persistent challenges in GovTech benchmarking, such as resource-intensive methodologies that provide retrospective rather than real-time analysis, a lack of complexity that overlooks digital infrastructures and emerging technologies in favor of simpler metrics, and improper levels of aggregation that can render results less useful. Considering that benchmarks can significantly influence political outcomes and shape the development of GovTech services, refining benchmarking methodologies using LLMs can mitigate these issues and potentially improve the responsiveness and relevance of government actions that better serve societal needs. Using Design Science Research Methodology and Activity Theory, an artefact is developed that combines an LLM with Retrieval-Augmented Generation (RAG), fine-tuning and prompt-engineering. The artefact is used to operationalize the GTMI-benchmark by the World Bank. The development of a benchmark specifically tailored for operationalization by LLMs is proposed, with a preliminary design for an AI-Supported GovTech Index (AGTI) outlined.

Files

Msc_Thesis_C_Snoeij.pdf
(pdf | 14 Mb)
License info not available