Towards Automated Stance Detection in Congressional Hearings with Large Language Models
A. Nikolaidis (TU Delft - Electrical Engineering, Mathematics and Computer Science)
S. Tan – Mentor (TU Delft - Interactive Intelligence)
E. Salas Gironés – Graduation committee member (TU Delft - Interactive Intelligence)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
U.S. congressional hearing transcripts offer a valuable window into national policy discourse, but they are prohibitively large for manual analysis. This study explores the use of large language models (LLMs) for multi-speaker, multi-target stance detection, a task that involves identifying each speaker's position on multiple topics within a single hearing. To this end, a novel annotation framework is introduced to produce stance labels for a small corpus of hearings from the House Oversight and Government Reform Committee. The study then evaluates the classification performance of zero- and few-shot prompting and investigates how chain-of-thought reasoning influences the results. The evaluation is conducted using OpenAI's GPT-4o and o3 models. Initial experimental results indicate that combining chain-of-thought with few-shot prompting yields the highest performance, suggesting a promising direction for automating stance analysis using LLMs in complex political discourse.