A System for Model Diagnosis centered around Human Computation

Machine learning (ML) systems for computer vision applications are widely deployed in decision-making contexts, including high-stakes domains such as autonomous driving and medical diagnosis. While largely accelerating the decision-making process, those systems have been found to suffer from a severe issue of reliability, i.e., they can easily...

To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems

Powerful predictive AI systems have demonstrated great potential in augmenting human decision-making. Recent empirical work has argued that the vision for optimal human-AI collaboration requires ‘appropriate reliance’ of humans on AI systems. However, accurately estimating the trustworthiness of AI advice at the instance level is quite...

Automatically Testing a Conversational Crowd Computing Platform

The rise in the use of crowd computing platforms led to the birth of Dandelion, a conversational crowd computing platform developed at TU Delft with the main goals being to connect students with researchers and to allow students to report on their well-being by using a friendly interface. Dandelion was tested manually up to the time of drafting...

Enterprise Crowd Computing for Human Aided Chatbots

chatbot is an example of cognitive computing system that emulates human conversations to provide informational, transactional,<br/>and conversational services. Despite their widespread adoption, chatbots still suffer from a number of performance issue due to limitations with their programming and training. In this paper we discuss Human Aided...