Designing Search-as-Learning Systems

Abstract

At the core of Search-as-Learning (SAL), as a sub-field of Interactive Information Retrieval (IIR), is exploring how people use search engines to acquire knowledge. Inherently interactive, the knowledge acquisition process via a search system involves learners posing queries, analyzing content, and incorporating novel knowledge. Being this iterative process, learning-oriented systems should be designed to support learners in their search process. In this thesis, we study how to design such systems by leveraging concepts from Natural Language Processing (NLP), Information Retrieval (IR), and the learning sciences.
We begin this thesis by proposing interventions in a search system to better support learners in their journey with ideas based on instructional scaffolding from the learning sciences. The data collected from this user study gives us insights into how learners interact with search systems and provides us with a rich dataset for the rest of the thesis. With that data available, the rest of this thesis focuses on modeling learner behavior using different paradigms and frameworks. We begin by simulating learners’ behavior, proposing a novel searcher model based on the idea that learning topics are usually subdivided into subtopics. Then, we propose a framework for tracking and predicting a learner’s knowledge state throughout their search session. By using only the information in the documents read by the learner, this framework can accurately predict the learner’s knowledge at the end of their session.
Finally, we look into SAL using the underexplored perspective of causality. While most previous works, including our own, mainly look into the correlations between behavior and learning, we take PLS-SEM, a causal modeling technique, to study the causal relationships between the different variables involved in the learning process and how this intricate and complex interactive process unfolds. From this analysis, we not only identify some interesting causal relationships, such as the indirect impact of query quality on learning but also show that the common practice of assessing learning by multiple-choice questions does not fully capture the variability of the learning process.
This thesis, therefore, contributes to the field of Search-as-Learning by making all of our datasets and code for simulating and modeling learner behavior available. We also provide several insights into how learners behave while searching and the impact of these behaviors on their learning outcomes. We hope the findings in this thesis can be used as foundations for more principled improvements in learning-oriented search systems.