document

Lower limb biomechanics of fully trained exoskeleton users reveal complex mechanisms behind the reductions in energy cost with human-in-the-loop optimization

Poggensee, K. (author), Collins, Steven H. (author)

Exoskeletons that assist in ankle plantarflexion can improve energy economy in locomotion. Characterizing the joint-level mechanisms behind these reductions in energy cost can lead to a better understanding of how people interact with these devices, as well as to improved device design and training protocols. We examined the biomechanical...

journal article 2024

document

Editorial: Special Issue on Human in the Loop Data Curation

Demartini, Gianluca (author), Sadiq, Shazia (author), Yang, J. (author)

This Special Issue of the Journal of Data and Information Quality (JDIQ) contains novel theoretical and methodological contributions on data curation involving humans in the loop. In this editorial, we summarize the scope of the issue and briefly describe its content.

contribution to periodical 2024

document

Data Model for Computer Vision Explainability, Fairness, and Robustness

Karnani, Simran (author)

In recent years, there has been a growing interest among researchers in the explainability, fairness, and robustness of Computer Vision models. While studies have explored the usability of these models for end users, limited research has delved into the challenges and requirements faced by researchers investigating these requirements. This study...

master thesis 2023

document

A System for Model Diagnosis centered around Human Computation

Ziad Ahmad Saad Soliman Nawar, Ziad (author)

Machine learning (ML) systems for computer vision applications are widely deployed in decision-making contexts, including high-stakes domains such as autonomous driving and medical diagnosis. While largely accelerating the decision-making process, those systems have been found to suffer from a severe issue of reliability, i.e., they can easily...

master thesis 2023

document

Supplementing Haptic Feedback in Flight Envelope Protection Through Visual Display Indications

de Rooij, G. (author), van Baelen, D. (author), Borst, C. (author), van Paassen, M.M. (author), Mulder, Max (author)

Haptic cues on the side stick are a promising method to reduce loss of control in-flight incidents. They can be intuitively interpreted and provide immediate support, leading to a shared control system. However, haptic interfaces are limited in providing information, and the reason for cues may not always be clear to pilots. This study presents...

journal article 2023

document

Predictive Control of a Human–in–the–Loop Network System Considering OperatorComfort Requirements

Sadowska, Anna D. (author), Maestre, José María (author), Kassking, Ruud (author), van Overloop, P.J.A.T.M. (author), De Schutter, B.H.K. (author)

We propose a model-predictive control (MPC)-based approach to solve a human-in-the-loop control problem for a network system lacking sensors and actuators to allow for a fully automatic operation. The humans in the loop are, therefore, essential; they travel between the network nodes to provide the remote controller with measurements and to...

journal article 2023

document

Perspective: Leveraging Human Understanding for Identifying and Characterizing Image Atypicality

Sharifi Noorian, S. (author), Qiu, S. (author), Sayin, Burcu (author), Balayn, A.M.A. (author), Gadiraju, Ujwal (author), Yang, J. (author), Bozzon, A. (author)

High-quality data plays a vital role in developing reliable image classification models. Despite that, what makes an image difficult to classify remains an unstudied topic. This paper provides a first-of-its-kind, model-agnostic characterization of image atypicality based on human understanding. We consider the setting of image classification...

conference paper 2023

document

Editorial: Human-centered AI: Crowd computing

Yang, J. (author), Bozzon, A. (author), Gadiraju, Ujwal (author), Lease, Matthew (author)

contribution to periodical 2023

document

How do you feel?: Measuring User-Perceived Value for Rejecting Machine Decisions in Hate Speech Detection

Lammerts, Philippe (author), Lippmann, P. (author), Hsu, Yen Chia (author), Casati, Fabio (author), Yang, J. (author)

Hate speech moderation remains a challenging task for social media platforms. Human-AI collaborative systems offer the potential to combine the strengths of humans' reliability and the scalability of machine learning to tackle this issue effectively. While methods for task handover in human-AI collaboration exist that consider the costs of...

conference paper 2023

document

Trust and Perceived Control in Burnout Support Chatbots

Degachi, C. (author), Al Owayyed, M. (author), Tielman, M.L. (author)

Increased levels of user control in learning systems is commonly cited as good AI development practice. However, the evidence as to the effect of perceived control over trust in these systems is mixed. This study investigated the relationship between different trust dimensions and perceived control in postgraduate student burnout support...

conference paper 2023

document

Rethinking the Role of AI with Physicians in Oncology: Revealing Perspectives from Clinical and Research Workflows

Verma, H. (author), Mlynar, Jakub (author), Schaer, Roger (author), Reichenbach, Julien (author), Jreige, Mario (author), Prior, John (author), Evéquoz, Florian (author), Depeursinge, Adrien (author)

Significant and rapid advancements in cancer research have been attributed to Artificial Intelligence (AI). However, AI’s role and impact on the clinical side has been limited. This discrepancy manifests due to the overlooked, yet profound, differences in the clinical and research practices in oncology. Our contribution seeks to scrutinize...

conference paper 2023

document

Predictive Control of Irrigation Canals Considering Well-being of Operators

Ranjbar, R. (author), Sadowska, A. (author), Maestre, J. M. (author), van Overloop, P.J.A.T.M. (author), De Schutter, B.H.K. (author)

We propose a model-predictive control (MPC) approach to solve a human-in-the-loop control problem for a non-automatic networked system with uncertain dynamics. There are no sensors or actuators installed in the system and we involve humans in the loop to travel between various nodes in the network and to provide the remote controller with...

journal article 2023

document

A Human-In-the-Loop Framework to Assess Multimodal Machine Learning Models

Chen, Dina (author)

Recent works explain the DNN models that perform image classification tasks following the "attribution, human-in-the-loop, extraction" workflow. However, little work has looked into such an approach for explaining DNN models for language or multimodal tasks. To address this gap, we propose a framework that explains and assesses the model...

master thesis 2022

document

CHIME: Causal Human-In-the-Loop Model Explanations

Biswas, Shreyan (author)

Explaining the behaviour of Artificial Intelligence models has become a necessity. Their opaqueness and fragility are not tolerable in high-stakes domains especially. <br/>Although considerable progress is being made in the field of Explainable Artificial Intelligence, scholars have demonstrated limits and flaws of existing approaches:...

master thesis 2022

document

Debugging Machine Learning on Time-series Data

Zheng, Meng (author)

Machine learning models are so-called a "black box," which means people can not easily observe the relationship between the output and input or explain the reason for such results. In recent years, much work has been done on interpretable machine-learning, such as Shapley values, counterfactual explanations, partial dependence plots, or saliency...

master thesis 2022

document

Characterizing the knowns and unknowns of text simplification models

WANG, Siwei (author)

master thesis 2022

document

A Human-In-the-Loop System for Interpreting Image Recognition Models

Ziengs, Bart (author)

Interpretability of ML models and image recognition models specifaclly, is a increasing problem. In this thesis, the design and implementation of Brickroutine: a system that used a trained model, is presented. Using human annotations, semantic interpretations are given to image classification problems. By giving an iterative approach in terms of...

master thesis 2022

document

Human-in-the-Loop Rule Discovery for Micropost Event Detection

Bhardwaj, Akansha (author), Yang, J. (author), Cudré-Mauroux, Philippe (author)

Platforms such as Twitter are increasingly being used for real-world event detection. Recent work often leverages event-related keywords for training machine learning based event detection models. These approaches make strong assumptions on the distribution of the relevant microposts containing the keyword – referred to as the expectation – and...

journal article 2022

document

CHIME: Causal Human-in-the-Loop Model Explanations

Biswas, S. (author), Corti, L. (author), Buijsman, S.N.R. (author), Yang, J. (author)

Explaining the behaviour of Artificial Intelligence models has become a necessity. Their opaqueness and fragility are not tolerable in high-stakes domains especially. Although considerable progress is being made in the field of Explainable Artificial Intelligence, scholars have demonstrated limits and flaws of existing approaches: explanations...

conference paper 2022

document

What Should You Know? A Human-In-the-Loop Approach to Unknown Unknowns Characterization in Image Recognition

Sharifi Noorian, S. (author), Qiu, S. (author), Gadiraju, Ujwal (author), Yang, J. (author), Bozzon, A. (author)

Unknown unknowns represent a major challenge in reliable image recognition. Existing methods mainly focus on unknown unknowns identification, leveraging human intelligence to gather images that are potentially difficult for the machine. To drive a deeper understanding of unknown unknowns and more effective identification and treatment, this...

conference paper 2022