Title
Explainable Cross-Topic Stance Detection for Search Results
Author
Draws, T.A. (TU Delft Web Information Systems) 
Natesan Ramamurthy, Karthikeyan (IBM Research)
Baldini, Ioana (IBM Research)
Dhurandhar, Amit (IBM Research)
Padhi, Inkit (IBM Research)
Timmermans, Benjamin (IBM Benelux)
Tintarev, N. (Universiteit Maastricht)
Date
2023
Abstract
One way to help users navigate debated topics online is to apply stance detection in web search. Automatically identifying whether search results are against, neutral, or in favor could facilitate diversification efforts and support interventions that aim to mitigate cognitive biases. To be truly useful in this context, however, stance detection models not only need to make accurate (cross-topic) predictions but also be sufficiently explainable to users when applied to search results - an issue that is currently unclear. This paper presents a study into the feasibility of using current stance detection approaches to assist users in their web search on debated topics. We train and evaluate 10 stance detection models using a stance-annotated data set of 1204 search results. In a preregistered user study (N = 291), we then investigate the quality of stance detection explanations created using different explainability methods and explanation visualization techniques. The models we implement predict stances of search results across topics with satisfying quality (i.e., similar to the state-of-the-art for other data types). However, our results reveal stark differences in explanation quality (i.e., as measured by users' ability to simulate model predictions and their attitudes towards the explanations) between different models and explainability methods. A qualitative analysis of textual user feedback further reveals potential application areas, user concerns, and improvement suggestions for such explanations. Our findings have important implications for the development of user-centered solutions surrounding web search on debated topics.
Subject
bias
explainability
stance detection
viewpoint
web search
To reference this document use:
http://resolver.tudelft.nl/uuid:a400cb4f-97ab-4c1b-9ab4-c76b2af79517
DOI
https://doi.org/10.1145/3576840.3578296
Publisher
Association for Computing Machinery (ACM), New York, NY, USA
ISBN
979-8-4007-0035-4
Source
CHIIR 2023 - Proceedings of the 2023 Conference on Human Information Interaction and Retrieval
Event
8th ACM SIGIR Conference on Human Information Interaction and Retrieval, CHIIR 2023, 2023-03-19 → 2023-03-23, Austin, United States
Series
CHIIR 2023 - Proceedings of the 2023 Conference on Human Information Interaction and Retrieval
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2023 T.A. Draws, Karthikeyan Natesan Ramamurthy, Ioana Baldini, Amit Dhurandhar, Inkit Padhi, Benjamin Timmermans, N. Tintarev