Human input is pivotal in building AI systems. Aiding the gathering of high-quality and representative human input on demand, microtask crowdsourcing platforms have thrived. Despite the benefits available, the lack of health provisions, safeguards, and existing practices threaten the sustainability of crowd work. Prior work investigated the usefulness of a dual-purpose input modality of ergonomically-informed gestures across different microtasks, finding that gestures as inputs offer a realistic trade-off between worker accuracy and potential short to long-term health benefits. However, little is understood about the effect of switching input modalities from one task to another on worker experiences and task-related outcomes. Addressing this research and empirical gap, we conducted a between-subjects study (N = 717) with varying sequences of input modalities across 16 experimental conditions to systematically understand the effect of switching input modalities. We found that the order of the input modality can influence the time it takes to complete tasks but does not affect accuracy. Further, the cognitive load perceived by workers was not significantly different between conditions. Our findings hint that ergonomically informed gestures can be effectively intertwined with conventional input modalities without a detrimental impact on worker experiences and quality-related outcomes. Our work has important implications for the design of human-centered crowdsourcing platforms that cater to worker health and wellbeing.

Crowdsourcing is a valuable tool to gather human input which enables the development of reliable artificial intelligence systems. Microtask platforms like Prolific and Amazon's Mechanical Turk have flourished by creating environments where crowd workers can provide such human input in a diverse and representative manner. Such marketplaces have evolved to support several hundreds of workers in earning their primary livelihood through crowd work. Crowd workers, however, often perform these tasks in sub-optimal work environments with poor ergonomics. Additionally, many of the various microtasks require input via the standard method of a mouse and keyboard and are repetitive in nature. As such, crowd workers who primarily earn their livelihoods in microtask marketplaces are at risk of injuries such as carpal tunnel syndrome. By changing the input modality from a mouse and keyboard to gesture-driven input, crowd workers can complete their work while simultaneously improving or safeguarding their physical health. Through three distinct microtasks, we constructed a dataset that enables the exploration of the physical and mental health of crowd workers while using gestures. In this work, we present the process of constructing this dataset, how we applied it, and the future applications we foresee.

Time constraints are commonplace in our daily lives. While literature in recent years from the Information Retrieval (IR) community has increased our understanding of the effects of time constraints on search, practical effects on search outcomes have rarely been evaluated. Little is known about how different search interfaces influence search outcomes and experiences in time-constrained search. This constitutes a knowledge gap that we aim to address in our work. Through a pre-registered 4x4 between-subjects crowdsourced user study, we investigate the influence of four different interfaces (list view, grid-based view, absence of result snippets, and linear scanning pattern view) on search outcomes and experiences under imposed time constraints (no constraint and constraints at two, five, and eight minutes). Results from our study indicate that user task performance is considerably affected by time constraints. In addition, as time constraints are tightened, a trade-off between querying rates and click depths arises. While no interaction effects between SERP interfaces and time constraints were ultimately found, findings from this study form an essential foundation for future work on how search result presentation may assist those searchers under strict time constraints.

We introduce a re-ranking model that augments the functionality of standard search engines to aid classroom search activities for children (ages 6–11). This model extends the known listwise learning-to-rank framework by balancing risk and reward. Doing so enables the model to prioritize Web resources of high educational alignment, appropriateness, and adequate readability by analyzing the URLs, snippets, and page titles of Web resources retrieved by a mainstream search engine. Experimental results demonstrate the value of considering multiple perspectives inherent to the classroom when designing algorithms that can better support children's information discovery.

Conversational User Interfaces (CUIs) have been argued to have advantages over traditional GUIs due to having a more human-like interaction. The growing popularity of conversational agents has enabled humans to interact with machines more naturally. There is an increasing familiarity among people with conversational interactions mediated by technology due to the widespread use of mobile devices and messaging services and a hungry market for conversational agents. Based on the recent advances in conversational AI, as a result of the proliferation of large language models, the signs are that the future of human-computer interaction will have a significant conversational component. Today, over two-thirds of the population on our planet has access to the Internet, with ever-lowering barriers to accessibility. This tutorial will showcase the benefits of employing novel conversational interfaces for crowd computing, human-AI decision making, health and well-being, and information retrieval. Given the widespread adoption of AI systems across several domains, we will discuss the potential of conversational interfaces in facilitating and mediating people's interactions with AI systems. The tutorial will include interactive elements and discussions and provide participants with insights to inform the design of effective conversational interfaces.

Human input is pivotal in building reliable and robust artificial intelligence systems. By providing a means to gather diverse, high-quality, representative, and cost-effective human in put on demand, micro task crowdsourcing marketplace shave thrived. Despite the unmistakable benefits available from online crowd work, the lack of health provisions and safeguards, along with existing work practices threatens the sustainability of this paradigm. Prior work has investigated worker engagement and mental health, yet no such investigations into the effects of crowd work on the physical health of workers have been undertaken. Crowd workers complete their work in various sub-optimal work environments, often using a conventional input modality of a mouse and keyboard. The repetitive nature of micro task crowdsourcing can lead to stress-related injuries, such as the well-documented carpal tunnel syndrome. It is known that stretching exercise scan help reduce injuries and discomfort in office workers. Gestures, the act of using the body intentionally to affect the behavior of an intelligent system, can serve as both stretches and an alternative form of input for micro tasks. To better understand the usefulness of the dual-purpose in put modality of ergonomically-informed gestures across different crowd sourced micro tasks, we carried out a controlled 2 × 3 between-subjects study (N=294). Considering the potential benefits of gestures as an input modality, our results suggesta real trade-off between worker accuracy in exchange for potential short to long-term health benefits.

Readability is a core component of information retrieval (IR) tools as the complexity of a resource directly affects its relevance: a resource is only of use if the user can comprehend it. Even so, the link between readability and IR is often overlooked. As a step towards advancing knowledge on the influence of readability on IR, we focus on Web search for children. We explore how traditional formulas–which are simple, efficient, and portable–fare when applied to estimating the readability of Web resources for children written in English. We then present a formula well-suited for readability estimation of child-friendly Web resources. Lastly, we empirically show that readability can sway children’s information access. Outcomes from this work reveal that: (i) for Web resources targeting children, a simple formula suffices as long as it considers contemporary terminology and audience requirements, and (ii) instead of turning to Flesch-Kincaid–a popular formula–the use of the “right” formula can shape Web search tools to best serve children. The work we present herein builds on three pillars: Audience, Application, and Expertise. It serves as a blueprint to place readability estimation methods that best apply to and inform IR applications serving varied audiences.

Music is an important part of childhood development, with online music listening platforms being a significant channel by which children consume music. Children's offline music listening behavior has been heavily researched, yet relatively few studies explore how their behavior manifests online. In this paper, we use data from LastFM 1 Billion and the Spotify API to explore online music listening behavior of children, ages 6-17, using education levels as lenses for our analysis. Understanding the music listening behavior of children can be used to inform the future design of recommender systems.

In this paper, we take a step towards understanding how to design search engine results pages (SERP) that encourage children's engagement as they seek for online resources. For this, we conducted a participatory design session to enable us to elicit children's preferences and determine what children (ages 6-12) find lacking in more traditional SERP. We learned that children want more dynamic means of navigating results and additional ways to interact with results via icons. We use these findings to inform the design of a new SERP interface, which we denoted CHIRP. To gauge the type of engagement that a SERP incorporating interactive elements-CHIRP-can foster among children, we conducted a user study at a public school. Analysis of children's interactions with CHIRP, in addition to responses to a post-task survey, reveals that adding additional interaction points results in a SERP interface that children prefer, but one that does not necessarily change engagement levels through clicks or time spent on SERP.

We present pathways of investigation regarding conversational user interfaces (CUIs) for children in the classroom. We highlight anticipated challenges to be addressed in order to advance knowledge on CUIs for children. Further, we discuss preliminary ideas on strategies for evaluation.

Past and current research has typically focused on ensuring that search technology for the classroom serves children. In this paper, we argue for the need to broaden the research focus to include teachers and how search technology can aid them. In particular, we share how furnishing a behind-the-scenes portal for teachers can empower them by providing a window into the spelling, writing, and concept connection skills of their students.

In this paper, we present BiGBERT, a deep learning model that simultaneously examines URLs and snippets from web resources to determine their alignment with children’s educational standards. Preliminary results inferred from ablation studies and comparison with baselines and state-of-the-art counterparts, reveal that leveraging domain knowledge to learn domain-aligned contextual nuances from limited input data leads to improved identification of educational web resources.

We present the preliminary exploration we conducted to identify traits that can influence children's preferences in books. Findings offer insights for the design of recommender algorithms that would look beyond patterns inferred from traditional user-system interactions (e.g., ratings) for recommendation purposes, since when it comes to children such data is rarely, if at all, available.