Living in the Age of AI: Understanding Contextual Factors that Shape Human-AI Decision-Making

Abstract

Decision-making has become increasingly intertwined with the use of AI systems to augment human capabilities. The primary goal of human-AI collaboration is to enhance outcomes by leveraging the strengths of both parties. However, as AI systems become more sophisticated, the relationship between humans and AI in decision-making has grown more complex, presenting both challenges and opportunities. For instance, the integration of AI systems can lead to over-reliance, reduced critical thinking, and sub-optimal decision-making outcomes, deviating from the intended benefits. The error rate and biases of AI systems, as well as the user's misunderstanding of the system's capabilities and limitations, can negatively impact the decision-making process and outcomes. To address these issues, researchers have examined various factors that can shape human decision-making behavior and outcomes, including human-related attributes (e.g., cognitive biases, individual differences, expertise), features of AI systems (e.g., transparency, explainability), and contextual factors (e.g., task complexity, uncertainty, time pressure). While many studies have been focused on the human-related factors and features of AI systems, less attention has been paid to the influence of contextual elements on human-AI decision-making.

This work contributes to the growing body of research on human-AI decision-making by empirically investigating the influence of contextual factors on decision-makers behaviors and outcomes. Through a series of studies, we demonstrate that factors such as task complexity, task uncertainty, and group dynamics can significantly impact the adoption of AI systems in decision-making contexts. Over-reliance on AI systems is more prevalent in complex and uncertain tasks, leading to sub-optimal outcomes and reduced critical thinking abilities. Additionally, we found that integrating AI systems can be more beneficial for groups than for individuals, as the collective intelligence and diverse perspectives within a group can enhance critical thinking and decision-making. These findings can inform the design of AI systems and the development of interventions that promote the appropriate use of AI in decision-making, tailored to the specific needs and characteristics of the context.

This thesis also informs the design of future empirical studies that aim to better understand the complex relationship between humans, AI systems, and the surrounding context. While it may not be practical to control all contextual factors in real-world settings, an awareness of their influence can guide the development of rigorous studies that can capture the dynamics of human-AI decision-making in realistic scenarios. Additionally, by proposing a configurable framework, this thesis provides a methodological toolset to enable future researchers to systematically investigate the various factors that contribute to the success of human-AI decision-making.