Influence of ChatGPT Expertise, Topical Expertise, and Topical Interest on User Behavior and Engagement in Informational Search Sessions in ChatGPT

Master thesis (2023)

Authors

T. Mo Electrical Engineering, Mathematics and Computer Science

Contributors

M.S. Pera Web Information Systems - EEMCS (mentor)
Ujwal Gadiraju Web Information Systems - EEMCS (mentor)
A. Rieger Web Information Systems - EEMCS (mentor)
M.L. Tielman Interactive Intelligence - EEMCS (graduation committee member)
Faculty
Electrical Engineering, Mathematics and Computer Science, Electrical Engineering, Mathematics and Computer Science
Copyright: © 2023 Tianyu Mo
More Info
expand_more

Published Date

11-12-2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

ChatGPT, a cutting-edge technology based on LLM, demonstrated great potential in search tasks. While the importance and potential of ChatGPT are growing, the gap in the understanding of how users interact and engage in ChatGPT search remains open. Past research has extensively examined traditional information search, but there is a need for investigation into user behaviour and engagement in LLM contexts like ChatGPT. To address this gap, our study aims to examine the impact of ChatGPT expertise, topical expertise, and topical interest on user behaviour and user engagement in ChatGPT search as we assume these factors play an important role in shaping user interactions with ChatGPT. We conducted an experiment to investigate the answer by inviting users (N=198) via the crowdsourcing platform to communicate with the mock ChatGPT application and their interactions were recorded for subsequent analysis. Prior to and after their interaction with the mock ChatGPT application, users are requested to complete a questionnaire to gather information about their user profiles and quantify their engagement. Our finding indicates that ChatGPT expertise has a partial influence on user engagement in ChatGPT search, which may highlight the importance of AI expertise in shaping user interactions. Furthermore, our research also indicates that the Affinity for Technology Interaction (ATI) has an impact on user engagement, which underscores the importance of understanding the psychological aspects in the age of artificial intelligence. The results of this study will not only address the current knowledge gap in ChatGPT search but also provide valuable information on how to improve ChatGPT to enhance user interaction experience.