Exploring the Feasibility of Crowd-Powered Decomposition of Complex User Questions in Text-to-SQL Tasks

Natural Language Interfaces to Databases (NLIDB), also known as Text-to-SQL models, enable users with different levels of knowledge in Structured Query Language (SQL) to access relational databases without any programming effort. By translating natural languages into SQL query, not only do NLIDBs minimize the burden of memorizing the schema...

Great Chain of Agents: The Role of Metaphorical Representation of Agents in Conversational Crowdsourcing

Conversational agents are being widely adopted across several domains to serve a variety of purposes ranging from providing intelligent assistance to companionship. Recent literature has shown that users develop intuitive folk theories and a metaphorical understanding of conversational agents (CAs) due to the lack of a mental model of the...

A Checklist to Combat Cognitive Biases in Crowdsourcing

Recent research has demonstrated that cognitive biases such as the confirmation bias or the anchoring effect can negatively affect the quality of crowdsourced data. In practice, however, such biases go unnoticed unless specifically assessed or controlled for. Task requesters need to ensure that task workflow and design choices do not trigger...

Remote Work Aided by Conversational Agents

Due to the coronavirus pandemic, remote work from home has rapidly become a necessity around the world, drastically changing the potential landscape for the future of work. Over the last couple of decades, microtask crowdsourcing has emerged as a viable means of carrying out remote online work to earn one’s living — an alternative to traditional...

Just the right mood for HIT!: Analyzing the role of worker moods in conversational microtask crowdsourcing

Conversational agents are playing an increasingly important role in providing users with natural communication environments, improving outcomes in a variety of domains in human-computer interaction. Crowdsourcing marketplaces are simultaneously flourishing, and it has never been easier to acquire large-scale human input from online workers....

How Does Team Composition Affect Knowledge Gain of Users in Collaborative Web Search?

Studies in searching as learning (SAL) have revealed that user knowledge gain not only manifests over a long-term learning period, but also occurs in single short-term web search sessions. Though prior works have shown that the knowledge gain of collaborators can be influenced by user demographics and searching strategies in long-term...