Searching the web on debated topics, which are issues under active discussion and where individual opinions diverge, can be highly challenging. It requires users to approach their queries objectively, browse many resources, and accept a certain level of uncertainty, even if it conflicts with personal values. In particular, confirmation bias, the tendency to favor information that reinforces beliefs or attitudes, impacts how people gather and interpret information retrieved from search engines when searching on debated topics. It can have far-reaching effects, potentially leading to conflicts, extremism, and polarization.

In an innovative approach towards mitigating the negative effects of confirmation bias on web search on debated topics, we propose implementing a boosting intervention aimed at enhancing Intellectual Humility (IH) – an individual’s ability to acknowledge the fallibility of one’s own beliefs and the limits of one’s knowledge while remaining open to learning from others’ perspectives even if they differ from their own viewpoint. While previous research has highlighted the potential benefits of boosting IH as a means to mitigate confirmation bias, its impact on users’ search behavior has yet to be explored.

Our work bridges this gap through two randomized preregistered user studies, gaining valuable insights into the effectiveness of IH-boosting interventions in mitigating confirmation bias. In the first study, we assessed the effect of three boosting interventions with different levels of complexity on users’ context dependent IH. In the second study, we examined the effects of these interventions on web search behavior.

The first experiment successfully demonstrated the effectiveness of the interventions in boosting participants’ IH across all three treatment groups. However, applying these interventions to web search, no significant differences in search behavior were observed. Our exploratory findings reveal that both individual and environmental factors, including occupation, personal viewpoints, and search results order, shape the impact of IH-boosting interventions on online search behavior, with varying effects observed across different debated topics. We hope this study inspires, and is an initial basis for continued efforts to explore the multifaceted relationship between IH, information-seeking behavior, and responsible opinion formation, ultimately promoting a more informed and unbiased online discourse.

Search engines are used to gather and collect information. This interaction sometimes influences the user and changes their attitude towards a topic after such interaction. Prior work has shown that it is a complex endeavour to understand attitude change, as there are many things that can influence a user during a search session. One of the aspects that have not been researched is the influence of informedness. Therefore we examine the role of informedness in attitude change. To do that, we first defined what a well-informed user in the context of web search is. Then a user study is conducted to understand informedness’s role in attitude change. From the platform Prolific, we recruited participants (N=320) for the main user study. The experiment is a 3 x 3 (Informedness level x SERP viewpoint bias) factorial between-subjects study and it requires interaction with the provided search results. The participants were placed in one of three categories, ”Well Informed”, ”Mildly Informed”, and ”Uninformed”, based on their results on a knowledge questionnaire. We created 3 different conditions for the search engine results page (SERP). The SERP had a viewpoint-biased ranking that was either supportive, opposing or balanced towards the topic. There were two topics, abortion and obesity, to which the participants were assigned. Our findings showed that 37% of the participants with a valid submission had changed attitudes. The findings from the user study suggest that the level of informedness does not play a role in facilitating attitude change among users during a web search. This has implications for the design of information retrieval systems and web search experiences in general.

ChatGPT, a cutting-edge technology based on LLM, demonstrated great potential in search tasks. While the importance and potential of ChatGPT are growing, the gap in the understanding of how users interact and engage in ChatGPT search remains open. Past research has extensively examined traditional information search, but there is a need for investigation into user behaviour and engagement in LLM contexts like ChatGPT. To address this gap, our study aims to examine the impact of ChatGPT expertise, topical expertise, and topical interest on user behaviour and user engagement in ChatGPT search as we assume these factors play an important role in shaping user interactions with ChatGPT. We conducted an experiment to investigate the answer by inviting users (N=198) via the crowdsourcing platform to communicate with the mock ChatGPT application and their interactions were recorded for subsequent analysis. Prior to and after their interaction with the mock ChatGPT application, users are requested to complete a questionnaire to gather information about their user profiles and quantify their engagement. Our finding indicates that ChatGPT expertise has a partial influence on user engagement in ChatGPT search, which may highlight the importance of AI expertise in shaping user interactions. Furthermore, our research also indicates that the Affinity for Technology Interaction (ATI) has an impact on user engagement, which underscores the importance of understanding the psychological aspects in the age of artificial intelligence. The results of this study will not only address the current knowledge gap in ChatGPT search but also provide valuable information on how to improve ChatGPT to enhance user interaction experience.