The rapid progress of artificial intelligence (AI) will increase opportunities for humans and AI-driven technology to collaborate as teammates. This requires both partners to learn from interactions about the task, each other and the team (co-learning). Co-learning can be supported by enabling partners to share knowledge and experiences on the task and team level. This paper first analyzes the requirements regarding tasks and environments for co-learning. These requirements were subsequently implemented in a testbed: a human and intelligent robot jointly conducting an urban search and rescue task in a simplified task environment. We designed Learning Design Patterns (LDPs): interaction sequences intended to initiate and facilitate co-learning. Effects of LDPs on collaboration, knowledge and understanding, and team performance were experimentally evaluated using the testbed. In comparison to a previous study, participants appreciated the robot more, had more interaction and displayed more commitment. Results show evidence that the LDPs, in comparison with no interventions, initiated and improved learning of the human team member, in particular on knowledge development and understanding the partner. Better knowledge and understanding did, however, not also lead to better team performance. Implications for co-learning in human-AI teams and for learning-supporting interventions are discussed.

With the progress of Artificial Intelligence, intelligent agents are increasingly being deployed in tasks for which ethical guidelines and moral values apply. As artificial agents do not have a legal position, humans should be held accountable if actions do not comply, implying humans need to exercise control. This is often labeled as Meaningful Human Control (MHC). In this paper, achieving MHC is addressed as a design problem, defining the collaboration between humans and agents. We propose three possible team designs (Team Design Patterns), varying in the level of autonomy on the agent’s part. The team designs include explanations given by the agent to clarify its reasoning and decision-making. The designs were implemented in a simulation of a medical triage task, to be executed by a domain expert and an artificial agent. The triage task simulates making decisions under time pressure, with too few resources available to comply with all medical guidelines all the time, hence involving moral choices. Domain experts (i.e., health care professionals) participated in the present study. One goal was to assess the ecological relevance of the simulation. Secondly, to explore the control that the human has over the agent to warrant moral compliant behavior in each proposed team design. Thirdly, to evaluate the role of agent explanations on the human’s understanding in the agent’s reasoning. Results showed that the experts overall found the task a believable simulation of what might occur in reality. Domain experts experienced control over the team’s moral compliance when consequences were quickly noticeable. When instead the consequences emerged much later, the experts experienced less control and felt less responsible. Possibly due to the experienced time pressure implemented in the task or over trust in the agent, the experts did not use explanations much during the task; when asked afterwards they however considered these to be useful. It is concluded that a team design should emphasize and support the human to develop a sense of responsibility for the agent’s behavior and for the team’s decisions. The design should include explanations that fit with the assigned team roles as well as the human cognitive state.