Influence of ChatGPT Expertise, Topical Expertise, and Topical Interest on User Behavior and Engagement in Informational Search Sessions in ChatGPT
More Info
expand_more
Abstract
ChatGPT, a cutting-edge technology based on LLM, demonstrated great potential in search tasks. While the importance and potential of ChatGPT are growing, the gap in the understanding of how users interact and engage in ChatGPT search remains open. Past research has extensively examined traditional information search, but there is a need for investigation into user behaviour and engagement in LLM contexts like ChatGPT. To address this gap, our study aims to examine the impact of ChatGPT expertise, topical expertise, and topical interest on user behaviour and user engagement in ChatGPT search as we assume these factors play an important role in shaping user interactions with ChatGPT. We conducted an experiment to investigate the answer by inviting users (N=198) via the crowdsourcing platform to communicate with the mock ChatGPT application and their interactions were recorded for subsequent analysis. Prior to and after their interaction with the mock ChatGPT application, users are requested to complete a questionnaire to gather information about their user profiles and quantify their engagement. Our finding indicates that ChatGPT expertise has a partial influence on user engagement in ChatGPT search, which may highlight the importance of AI expertise in shaping user interactions. Furthermore, our research also indicates that the Affinity for Technology Interaction (ATI) has an impact on user engagement, which underscores the importance of understanding the psychological aspects in the age of artificial intelligence. The results of this study will not only address the current knowledge gap in ChatGPT search but also provide valuable information on how to improve ChatGPT to enhance user interaction experience.