Comparing User Approach to Complex Information Needs in Traditional Web Search and Conversational Search
L.G. Leibbrandt (TU Delft - Electrical Engineering, Mathematics and Computer Science)
C Hauff – Mentor (TU Delft - Web Information Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Conversational search systems have recently gained popularity due to their effectiveness in performing small tasks and answering factoid questions. However, under complex search scenarios, these systems fail and fall back to techniques used by traditional search engines. As tasks requiring user learning are inherently complex, user knowledge is a powerful indicator of system performance under complex search and user ability to successfully interact with the information content. We present a study in which user knowledge gain and query formulation is explored in both the traditional web search and conversational search formats. Through crowd sourcing, 50 participants were recruited and conducted complex search sessions on either the traditional or conversational search medium. Through the use of a knowledge test given to the participant both before and after the search session, knowledge gain was determined. Furthermore, session queries and timestamps were tracked. It is found that participants have a significantly higher knowledge gain in traditional search, while conversational search sessions tended to have a higher number of queries and average query length.