Improving User Engagement to Reduce Dropout Rates in Long Web Surveys: Exploring the Effectiveness of Achievement Primes Amongst Intrinsically and Extrinsically Motivated Respondents

Web surveys have increasingly been used to collect data from respondents over the years. They offer several advantages compared to other methods of obtaining data. Researchers benefit from a broad demographic representation to make generalized conclusions, and satisfaction surveys allow employees to explain shortcomings or improvements...

Annotation practices in affective computing: What are these algorithms actually trained on?

In the machine learning research community, significant importance is given to the optimization of techniques which are employed once a benchmark dataset is given. However, less importance is assigned to the quality of these datasets and to how these datasets are obtained. In this work, we look into annotation practices in the research area of...

Annotation Practices in Societally Impactful Machine Learning Applications: What are the recommender systems models actually trained on?

Machine Learning models are nowadays infused into all aspects of our lives. Perhaps one of its most common applications regards recommender systems, as they facilitate users' decision-making processes in various scenarios (e.g., e-commerce, social media, news, online learning, etc.). Training performed on large volumes of data is what ultimately...

Annotation Practices in Machine Learning Research On Depression

Depression diagnosis and treatment remain difficult tasks that could be improved with machine learning models. But those automatic systems should be reliable to apply in clinical psychology settings. Performing predictions in this field is most commonly done using supervised learning models, which rely on well-established annotations. Therefore...

Investigating Data Collection and Reporting Practices of Human Annotations in Societally Impactful Machine Learning Applications: A Systematic Review of Top-Cited IEEE Access Papers

This systematic review investigates the practices and implications of human annotations in machine learning (ML) research. Analyzing a selection of 100 papers from the IEEE Access Journal, the study explores the data collection and reporting methods employed. The findings reveal a prevalent lack of standardization and formalization in the...

A Quest through Interconnected Datasets: Research on Annotation Practices in Highly Cited Audio Machine Learning Work and Their Utilized Datasets: Annotation Practices in Datasets Utilized by The International Conference on Acoustics, Speech, and Signal Processing (ICASSP) Conferences: A Transparency Analysis

This research examines transparency between ICASSP conference papers and the dataset documentations related to the datasets' annotation practices. Top-cited 5 papers and 51 unique resources in total were considered. All of the selected papers utilized at least one dataset. For every paper, an extensive metadata search has done to reach the...

Human- or Robot-like Music Assistive Robots: Effects on Fluency and Memory Recall

Longer lifespans and an ageing population put tremendous pressure on the care of the elderly. With the technology of robotics breakthroughs, it appears that the use of robotics in elderly care is ready to take off. Interestingly, more and more robots are now being created with a human-like appearance and demeanour, yet this could lead to issues...

Using Sign-Language as an Input Modality for Microtask Crowdsourcing

Several input types have been developed in different technological landscapes like crowdsourcing and conversational agents. However, sign language remains one of the input types that has not been looked upon. Although numerous amount of people around the world use sign language as their primary language, there have not been many efforts to...

Closer or even farther from fairness: An assessment of whether fairness toolkits constrain practitioners with regards to algorithmic harms

To encourage ethical thinking in Machine Learning (ML) development, fairness researchers have created tools to assess and mitigate unfair outcomes. However, despite their efforts, algorithmic harms go beyond what the toolkits currently allow to measure. Through 30 semi-structured interviews, we investigated whether data scientists are...

Who Cares About Fairness: How Background Influences the Way Practitioners Consider Machine Learning Harms

The increasing dangers of unfairness in machine learning (ML) are becoming a frequent subject of discussion, both, in academia and popular media. Recent literature focused on introducing and assessing algorithmic solutions to bias in ML. However, there is a disconnect between these solutions and practitioners' needs. By interviewing 30 ML...

Comparison of the usage of Fairness Toolkits amongst practitioners:AIF360 and Fairlearn

Machine learning is still one of the most rapidly growing fields, and is used in a variety of different sectors such as education, healthcare, financial modeling etc(Jordan and Mitchell 2015). However, along with this demand for machine learning algorithms, there comes a need for ensuring that these algorithms are fair and contain little to no...

Surfacing Differences in Practices When Building Fair Machine Learning Systems with Fairness Toolkits: an Empirical Study

The ability to identify and mitigate various risks and harms of using Machine Learning models in industry is an essential task. Specifically because these may produce harmful outcomes for stakeholders, including unfair or discriminatory results. Due to this there has been substantial research into the concepts of fairness and its metrics, bias...

A Study of Bugs Found in the Puppet Configuration Management System

This research studies the symptoms, root causes, impact, triggers, fixes, and system dependency of bugs in the Puppet configuration management system. Puppet is a widely used open-source configuration management system that performs various administrative tasks on machines based on a central specification. This paper aims to fill the research...

Studying bugs in the Salt Configuration Management System

Configuration management systems are a class of software used to automate system administrative tasks, one of which is the configuration of software systems. Although the automation is less error-prone than manual configuration done by a human, bugs in the source code can still cause configuration errors. This can result into unwanted...

A Study of Bugs Found in the Moby Configuration Management System

The study of bugs can provide important information to understand their nature in the context of complex software systems as well as supporting developers in their detection, fix and prevention. Previous studies focused on analyzing bugs under different perspectives such as changes at code level, frequency, semantics, symptoms, root causes and...

A study of bugs found in the Ansible configuration management system

Research that focuses on examining software bugs is critical when developing tools for preventing and for fixing software issues. Previous work in this area has explored other types of systems, such as bugs of compilers and security issues stemming from open source systems hosted on public repositories. This paper explores the bugs within the...

Offline-Online Reoptimization using a Hybrid Method

Realistic vehicle routing problems have been highly relevant for years in a wide variety of domains. One such domain is food delivery, where well-crafted routes can reduce costs and contribute to customer satisfaction. This thesis formulates a problem variant for the restaurant meal delivery problem in order to examine the reoptimization of meal...