Collaboration in teams composed of both humans and automations has an interdependent nature, which demands calibrated trust among all the teammembers. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, automations occasionally fail to do their job, which leads to a decrease in human’s trust. However, research has given contradictory statements about the effects of such a reduction of trust on the human’s trustworthiness, i.e. human’s characteristics that make them more or less reliable to the automation. As such, this study investigates how automation failure in a human-automation teamwork scenario affects the human’s trust in the automation and human’s trustworthiness towards the automation. We present a between-subjects controlled experiment in which the participants perform a simulated task in a 2D grid-world, collaborating with an automation in a “moving-out” scenario. During the experiment, we measure the participants’ trust and trustworthiness regarding the automation both subjectively and objectively. Our results show that automation failure negatively affects the human’s trustworthiness, as well as their trust in and liking of the automation. Learning the effects of automation failure in trust and trustworthiness can contribute to a better understanding of the nature and dynamics of trust in these teams, foreseeing undesirable consequences and improving human-automation teamwork.

OKademy is a start-up that wants to improve the healthcare in the Netherlands by improving the process by which graduated medical students are being matched to a hospital team. This is needed since there is a lack of surgery
assistants, while graduate students need to wait on average six months to start their work in a hospital. To match quickly and sufficiently, the hard-skills as well as soft-skills of the student need to be known. To achieve this, universities need to provide OKademy with each student’s soft-skills. However, this takes an excessive amount of effort for the universities. To let the universities benefit from this idea, OKademy wants to have a system which keeps track of the soft-skills and matches students in groups for assignments, thus being beneficial for universities and hospitals. To help Okademy achieve its goal we developed a system that can be used by universities to help the students keep track of their soft-skills. The same system can be used to form theoretically well-functioning groups. These groups should be generated using an optimization algorithm that approximates the best groups regarding the soft-skills of students. In this system, students, referred to as ‘members’, and instructors, referred to as ‘hosts’, can register. Members can be assigned into groups and hosts can create or modify groups and courses accordingly. Members can, after registration, fill in their top 10 soft-skills with a corresponding automatized grade. Based on these skills, they will be assigned to groups for a course created by a host. A course will go through multiple phases. When all groups have completed the assignment, a member is able to give feedback to five random soft-skills of their group members and recommend a new one if desired. Based on this feedback, their soft-skill grades will be adjusted. The possible combinations of groups can be extremely large, therefore, it is not feasible to blindly search for the best group formation. Our algorithm will use the idea of genetic algorithms to explore the search space and approximate the best group formation. The the final product consists of a web application in which the multi-objective group formation optimization algorithm is implemented. To work in a structured manner, we used the Scrum method. Meetings with our client and coach were held every week. The system has been tested by functionality tests, user tests and unit tests, ensuring a functional system. OKademy will use this system in collaboration with universities and hospitals to solve the problem.

Artificial Intelligence (AI) is increasingly affecting people’s lives. AI is even employed in fields where human lives depend on the AI’s decisions. However, these algorithms lack transparency, i.e. it is unclear how they determine the outcome. If, for instance, the AI’s purpose is to classify an image, the AI will learn this from examples provided to it (e.g. an image of a cow in a meadow). The algorithm can focus on the wrong part of the image. Instead of focusing on the foreground (cow), it could focus on the background (meadow). This way, by focusing on the background, it could produce a false output (e.g. a horse instead of a cow). To show this, an explanation is needed. For this reason, a variety of methods have been created to explain the reasoning behind these algorithms, called explainability methods. In this paper, six local explainability methods are discussed and compared. These methods were chosen as they were the most prominently used approaches for explainability methods for Convolutional Neural Networks (CNN). By comparing methods with analogous characteristics, this paper is going to show what methods exceed others in terms of performance. Furthermore, their advantages and limitations are being discussed. The comparison shows that Local Interpretable Model-agnostic Explanations, Layer-wise Relevance Propagation and Gradient-weighted Class Activation Mapping perform better than Sensitivity Analysis, Deep Taylor Decomposition and Deconvolutional Network, respectively.