Robots are now pervasive, leveraging their automation capabilities to assist humans across a diverse range of tasks. Nevertheless, end-users may have a limited understanding of the robot’s operation and typically assume a passive role when interacting with the robot performing a particular task. In this study, we address the critical need for effective explainability in human-robot interaction. By comparing different methods of explaining robotic scenario information to end-users, the proposed methodologies use a labelled property graph-based chatbot that adheres to the IEEE Robotics Ontology Standards. In this study, we designed two virtual robotic scenarios and simulated their information flow using the Robot Operating System. A between-subjects experiment was conducted where participants engaged with the system through various interaction methods to understand the two scenarios. These methods included real-time Linux Command Line Interface outputs, querying a chatbot, exploring knowledge graphs, or a combination of chatbot and knowledge graphs. The study findings suggest that both the knowledge graphs and the chatbot significantly enhance the system’s explainability compared to a simple Linux terminal information output. Moreover, utilizing knowledge graphs alongside the chatbot has received better subjective evaluations concerning metrics such as clarity, usability, and robustness. This research made contributions towards the development of standardised labelled property graphs for representing scenario information in language-based human-robot interaction. The experiment design and evaluations also provided a solution for assessing the explainability of task-oriented dialogue systems both subjectively and objectively.

While consciousness has been historically a heavily debated topic, awareness had less success in raising the interest of scholars. However, more and more researchers are getting interested in answering questions concerning what awareness is and how it can be artificially generated. The landscape is rapidly evolving, with multiple voices and interpretations of the concept being conceived and techniques being developed. The goal of this paper is to summarize and discuss the ones among these voices connected with projects funded by the EIC Pathfinder Challenge “Awareness Inside” callwithin Horizon Europe, designed specifically for fostering research on natural and synthetic awareness. In this perspective, we dedicate special attention to challenges and promises of applying synthetic awareness in robotics, as the development of mature techniques in this new field is expected to have a special impact on generating more capable and trustworthy embodied systems.

We present a sampling-based model predictive control method that uses a generic physics simulator as the dynamical model. In particular, we propose a Model Predictive Path Integral controller (MPPI) that employs the GPU-parallelizable IsaacGym simulator to compute the forward dynamics of the robot and environment. Since the simulator implicitly defines the dynamic model, our method is readily extendable to different objects and robots, allowing one to solve complex navigation and contact-rich tasks. We demonstrate the effectiveness of this method in several simulated and real-world settings, including mobile navigation with collision avoidance, non-prehensile manipulation, and whole-body control for high-dimensional configuration spaces. This is a powerful and accessible open-source tool to solve many contact-rich motion planning tasks.

Task and Motion Planning (TAMP) has made strides in complex manipulation tasks, yet the execution robustness of the planned solutions remains overlooked. In this work, we propose a method for reactive TAMP to cope with runtime uncertainties and disturbances. We combine an Active Inference planner (AIP) for adaptive high-level action selection and a novel Multi-Modal Model Predictive Path Integral controller (M3P2I) for low-level control. This results in a scheme that simultaneously adapts both high-level actions and low-level motions. The AIP generates alternative symbolic plans, each linked to a cost function for M3P2I. The latter employs a physics simulator for diverse trajectory rollouts, deriving optimal control by weighing the different samples according to their cost. This idea enables blending different robot skills for fluid and reactive plan execution, accommodating plan adjustments at both the high and low levels to cope, for instance, with dynamic obstacles or disturbances that invalidate the current plan. We have tested our approach in simulations and real-world scenarios.

In this article, we propose a hybrid combination of active inference and behavior trees (BTs) for reactive action planning and execution in dynamic environments, showing how robotic tasks can be formulated as a free-energy minimization problem. The proposed approach allows handling partially observable initial states and improves the robustness of classical BTs against unexpected contingencies while at the same time reducing the number of nodes in a tree. In this work, we specify the nominal behavior offline, through BTs. However, in contrast to previous approaches, we introduce a new type of leaf node to specify the desired state to be achieved rather than an action to execute. The decision of which action to execute to reach the desired state is performed online through active inference. This results in continual online planning and hierarchical deliberation. By doing so, an agent can follow a predefined offline plan while still keeping the ability to locally adapt and take autonomous decisions at runtime, respecting safety constraints. We provide proof of convergence and robustness analysis, and we validate our method in two different mobile manipulators performing similar tasks, both in a simulated and real retail environment. The results showed improved runtime adaptability with a fraction of the hand-coded nodes compared to classical BTs.

Once deployed in the real world, autonomous underwater vehicles (AUVs) are out of reach for human supervision yet need to take decisions to adapt to unstable and unpredictable environments. To facilitate research on self-adaptive AUVs, this paper presents SUAVE, an exemplar for two-layered system-level adaptation of AUVs, which clearly separates the application and self-adaptation concerns. The exemplar focuses on a mission for underwater pipeline inspection by a single AUV, implemented as a ROS 2-based system. This mission must be completed while simultaneously accounting for uncertainties such as thruster failures and unfavorable environmental conditions. The paper discusses how SUAVE can be used with different self-adaptation frameworks, illustrated by an experiment using the Metacontrol framework to compare AUV behavior with and without self-adaptation. The experiment shows that the use of Metacontrol to adapt the AUV during its mission improves its performance when measured by the overall time taken to complete the mission or the length of the inspected pipeline.

Self-adaptation can be used in robotics to increase system robust- ness and reliability. This work describes the Metacontrol method for self-adaptation in robotics. Particularly, it details how the MROS (Metacontrol for ROS Systems) framework implements and pack- ages Metacontrol, and it demonstrate how MROS can be applied in a navigation scenario where a mobile robot navigates in a factory floor. Video: https://www.youtube.com/watchvISe9aMskJuE

This work presents a fault-tolerant control scheme for sensory faults in robotic manipulators based on active inference. In the majority of existing schemes a binary decision of whether a sensor is healthy (functional) or faulty is made based on measured data. The decision boundary is called a threshold and it is usually deterministic. Following a faulty decision, fault recovery is obtained by excluding the malfunctioning sensor. We propose a stochastic fault-tolerant scheme based on active inference and precision learning which does not require a priori threshold definitions to trigger fault recovery. Instead, the sensor precision, which represents its health status, is learned online in a model-free way allowing the system to gradually, and not abruptly exclude a failing unit. Experiments on a robotic manipulator show promising results and directions for future work are discussed.

Nowadays smart applications appear in domains spanning from commodity household applications to advanced underwater robotics. These smart applications require adaptation to dynamic environments, changing requirements and internal system errors Metacontrol takes a systems of systems view on autonomous control systems and self-adaptation, by means of an additional layer of control that manipulates and combines the regular controllers. This paper develops a formal model of a Metacontrol architecture. We formalise this Metacontrol architecture in the context of an autonomous house heating application, enabling different controllers to be dynamically combined in order to meet user requirements to a better extent than the individual controllers in isolation. The formal model is developed in the Maude rewriting system, where we show results comparing different scenarios.

Known attempts to build autonomous robots rely on complex control architectures, often implemented with the Robot Operating System platform (ROS). The implementation of adaptable architectures is very often ad hoc, quickly gets cumbersome and expensive. Reusable solutions that support complex, runtime reasoning for robot adaptation have been seen in the adoption of ontologies. While the usage of ontologies significantly increases system reuse and maintainability, it requires additional effort from the application developers to translate requirements into formal rules that can be used by an ontological reasoner. In this paper, we present a design tool that facilitates the specification of reconfigurable robot skills. Based on the specified skills, we generate corresponding runtime models for self-adaptation that can be directly deployed to a running robot that uses a reasoning approach based on ontologies. We demonstrate the applicability of the tool in a real robot performing a patrolling mission at a university campus.

Autonomous systems are expected to maintain a dependable operation without human intervention. They are intended to fulfill the mission for which they were deployed, properly handling the disturbances that may affect them. Underwater robots, such as the UX-1 mine explorer developed in the UNEXMIN project, are paradigmatic examples of this need. Underwater robots are affected by both external and internal disturbances that hamper their capability for autonomous operation. Long-term autonomy requires not only the capability of perceiving and properly acting in open environments but also a sufficient degree of robustness and resilience so as to maintain and recover the operational functionality of the system when disturbed by unexpected events. In this article, we analyze the operational conditions for autonomous underwater robots with a special emphasis on the UX-1 miner explorer. We then describe a knowledge-based self-awareness and metacontrol subsystem that enables the autonomous reconfiguration of the robot subsystems to keep mission-oriented capability. This resilience augmenting solution is based on the deep modeling of the functional architecture of the autonomous robot in combination with ontological reasoning to allow self-diagnosis and reconfiguration during operation. This mechanism can transparently use robot functional redundancy to ensure mission satisfaction, even in the presence of faults.

We present a fault tolerant control scheme for robot manipulators based on active inference. The proposed solution makes use of the sensory prediction errors in the free-energy to simplify the residuals and thresholds generation for fault detection and isolation and does not require additional controllers for fault recovery. Results validating the benefits in a simulated 2DOF manipulator are presented and the limitations of the current approach are highlighted.

In this chapter we take a systems engineering stand to perform a postmortem analysis of the design of the robot motion subsystem in Team Delft’s robot winner of the Amazon Picking Challenge 2016, understanding the benefits and limitations of the motion planning approach taken. We use an analysis framework based on Model-Based Systems Engineering with the ISEandPPOOA methodology, and a novel model of levels of robot automation. The functional approach of ISEandPPOOA helps us understand how the design decisions in the architecture of the solution impact the performance and quality attributes in capabilities required for the competition. The levels of robot automation help analyze the fundamental properties of the control schemas applied at different levels of the control architecture when handling uncertainty.

Autonomous underwater robots, such as the UX-1 developed in the UNEXMIN project, need to maintain reliable autonomous operation in hazardous and unknown environments. Because of the lack of any kind of real-time communications with a human operated command and control station, the control architecture needs to be enhanced with mission-level self-diagnosis and self-adaptation properties an additional provided by some kind of supervisory or “metacontrol” component to ensure its reliability. In this paper, we propose an ontological implementation of such component based on Web Ontology Language (OWL) and the Semantic Web Rule Language (SWRL). The solution is based on an ontology of the functional architecture of autonomous robots, which allows inferring the effects of the performance of its constituents components in the functions required during the robot mission, and generate the reconfigurations needed to maintain operation reliably. The concept solution has been validated using a hypothetical set of scenarios implemented in an OWL ontology and an OWLAPI-based reasoner, which we aim at validating by integrating the metacontrol reasoning with a realistic simulation of the underwater robot.