Print Email Facebook Twitter On the Effects of Automatically Generated Adjunct Questions for Search as Learning Title On the Effects of Automatically Generated Adjunct Questions for Search as Learning Author Zhu, P. (TU Delft Web Information Systems) Câmara, Arthur (TU Delft Web Information Systems; Zeta Alpha) Roy, N. (TU Delft Web Information Systems) Maxwell, D.M. (TU Delft Web Information Systems; Booking.com B.V) Hauff, C. (TU Delft Web Information Systems; Spotify) Date 2024 Abstract Actively engaging learners with learning materials has been shown to be very important in the Search as Learning (SAL) setting. One active reading strategy relies on asking so-called adjunct questions, i.e., manually curated questions geared towards essential concepts of the target material. However, manual question creation is impractical given the vast online content. Recent research has explored the effects of Automatic Question Generation (AQG) on aiding human learning. These studies have primarily focused on user studies in controlled online reading scenarios with limited documents. However, the impacts of adjunct questions on learning in the SAL setting, which involves learning through web searching, are not yet well understood. This paper addresses this gap by conducting a user study with automatically generated adjunct questions integrated into the reading interface built on top of a search system. We conducted a between-subjects user study (N = 144) to investigate the incorporation of automatically generated adjunct questions on participants' learning. We employed three different question generation strategies as well as a control condition: (i) synthesis questions; (ii) factoid questions targeting random text spans; and (iii) factoid questions targeting terms and phrases relevant to the information need at hand. We present four major findings: (i) participants who received adjunct questions exhibited significantly more fine-grained reading behaviour, such as longer document dwell time and more scrolls, than those without adjunct questions. However, adjunct questions' influence on learning outcomes depends on the AQG strategy. (ii) Question types significantly influence participants' reading behaviour. (iii) The adjunct questions' target spans significantly influence learning outcomes. Lastly, (iv) participants' prior knowledge levels affect adjunct questions' effects on their learning outcomes and their reaction to different AQG strategies. Our findings have significant design implications for learning-oriented search systems. The data and code is available at https://github.com/zpeide/AQG-AdjunctQuestions. To reference this document use: http://resolver.tudelft.nl/uuid:ff5be262-5634-40d5-81e1-4b21a97bfc01 DOI https://doi.org/10.1145/3627508.3638332 Publisher Association for Computing Machinery (ACM) ISBN 9798400704345 Source CHIIR 2024 - Proceedings of the 2024 Conference on Human Information Interaction and Retrieval Event 2024 Conference on Human Information Interaction and Retrieval, CHIIR 2024, 2024-03-10 → 2024-03-14, Sheffield, United Kingdom Series CHIIR 2024 - Proceedings of the 2024 Conference on Human Information Interaction and Retrieval Part of collection Institutional Repository Document type conference paper Rights © 2024 P. Zhu, Arthur Câmara, N. Roy, D.M. Maxwell, C. Hauff Files PDF 3627508.3638332.pdf 2.15 MB Close viewer /islandora/object/uuid:ff5be262-5634-40d5-81e1-4b21a97bfc01/datastream/OBJ/view