Segmenting gas bubbles in multiphase flows is a critical yet unsolved challenge in numerous industrial settings, from metallurgical processing to maritime drag reduction. Traditional approaches — and most recent learning-based methods — assume near-spherical shapes, limiting their effectiveness in regimes where bubbles undergo deformation, coalescence, or breakup. This complexity is particularly evident in air lubrication systems, where coalesced bubbles form amorphous and topologically diverse patches. In this work, we revisit the problem through the lens of modern vision foundation models. We cast the task as a transfer learning problem and demonstrate, for the first time, that a fine-tuned Segment Anything Model (SAM v2.1) can accurately segment highly non-convex, irregular bubble structures using as few as 100 annotated images.

Learning from demonstration is a promising approach for teaching robots new skills. However, a central challenge in the execution of acquired skills is the ability to recognize faults and prevent failures. This is essential because demonstrations typically cover only a limited set of scenarios and often only the successful ones. During task execution, unforeseen situations may arise, such as changes in the robot's environment or interaction with human operators. To recognize such situations, this paper focuses on teaching the robot situational awareness by using a camera input and labeling frames as safe or risky. We train a Gaussian Process (GP) regression model fed by a low-dimensional latent space representation of the input images. The model outputs a continuous risk score ranging from zero to one, quantifying the degree of risk at each timestep. This allows for pausing task execution in unsafe situations and directly adding new training data, labeled by the human user. Our experiments on a robotic manipulator show that the proposed method can reliably detect both known and novel faults using only a single example for each new fault. In contrast, a standard multi-layer perceptron (MLP) performs well only on faults it has encountered during training. Our method enables the next generation of cobots to be rapidly deployed with easy-to-set-up, vision-based risk assessment, proactively safeguarding humans and detecting misaligned parts or missing objects before failures occur.

Quadrupedal animals show remarkable capabilities in traversing diverse terrains and display a range of behaviours and gait patterns. Achieving similar performance by exploiting the natural dynamics of the system is a key goal for robotics researchers. Here we show a bioinspired approach to the design of quadrupeds that seeks to exploit the body and the passive properties of the robot while maintaining active controllability on the system through minimal actuation. Utilizing an end-to-end computational design pipeline, neuromechanical couplings recorded in biological quadrupeds are translated into motor synergies, allowing minimal actuation to control the full structure via multijoint compliant mechanical couplings. Using this approach, we develop PAWS, a passive automata with synergies. By leveraging the principles of motor synergies, the design incorporates variable stiffness, anatomical insights and self-organization to simplify control while maximizing its capabilities. The resulting synergy-based quadruped requires only four actuators and exhibits emergent, animal-like dynamical responses, including passive robustness to environmental perturbations and a wide range of actuated behaviours. The finding contributes to the development of machine physical intelligence and provides robots with more efficient and natural-looking robotic locomotion by combining synergistic actuation, compliant body properties and embodied compensatory strategies.

Robotics is shifting from rigid, articulated systems to more sophisticated and heterogeneous mechanical structures. Soft robots, for example, have continuously deformable elements capable of large deformations. The flourishing of control techniques developed for this class of systems is fueling the need of efficient procedures for evaluating their inverse dynamics (ID), which is challenging due to the complex and mixed nature of these systems. As of today, no single ID algorithm can describe the behavior of generic (combinations of) models of soft robots. We address this challenge for generic series-like interconnections of possibly soft structures that may require heterogeneous modeling techniques. Our proposed algorithm requires as input a purely geometric description (forward-kinematics-like) of the mapping from configuration space to deformation space. With this information only, the complete equations of motion can be given an exact recursive structure which is essentially independent from (or “agnostic” to) the underlying reduced-order kinematics. We achieve this by exploiting Kane’s method to manipulate the equations of motion, showing then their recursive structure. The resulting ID algorithms have optimal computational complexity within the proposed setting, that is, linear in the number of distinct modules. Further, a variation of the algorithm is introduced that can evaluate the generalized mass matrix without increasing computation costs. We showcase the method applicability to robot models involving a mixture of rigid and soft elements, described via possibly heterogeneous reduced order models (ROMs), such as Volumetric FEM, Cosserat strain-based, and volume-preserving deformation primitives. None of these systems can be handled using existing ID techniques.

Model-based manipulation of deformable objects has traditionally dealt with objects while neglecting their dynamics, thus mostly focusing on very lightweight objects at steady state. At the same time, soft robotic research has made considerable strides toward general modeling and control, despite soft robots and deformable objects being very similar from a mechanical standpoint. In this work, we leverage these recent results to develop a control-oriented, fully dynamic framework of slender deformable objects grasped at one end by a robotic manipulator. We introduce a dynamic model of this system using functional strain parameterizations and describe the manipulation challenge as a regulation control problem. This enables us to define a fully model-based control architecture, for which we can prove analytically closed-loop stability and provide sufficient conditions for steady state convergence to the desired state. The nature of this work is intended to be markedly experimental. We provide an extensive experimental validation of the proposed ideas, tasking a robot arm with controlling the distal end of six different cables, in a given planar position and orientation in space.

While consciousness has been historically a heavily debated topic, awareness had less success in raising the interest of scholars. However, more and more researchers are getting interested in answering questions concerning what awareness is and how it can be artificially generated. The landscape is rapidly evolving, with multiple voices and interpretations of the concept being conceived and techniques being developed. The goal of this paper is to summarize and discuss the ones among these voices connected with projects funded by the EIC Pathfinder Challenge “Awareness Inside” callwithin Horizon Europe, designed specifically for fostering research on natural and synthetic awareness. In this perspective, we dedicate special attention to challenges and promises of applying synthetic awareness in robotics, as the development of mature techniques in this new field is expected to have a special impact on generating more capable and trustworthy embodied systems.

Soft robotics integrates engineering, materials science, and biology to tackle challenges that conventional robotics cannot solve. Alongside the advancements in soft robot technology, there is also a need for a standardized hardware platform that can enable benchmarking of various control methods developed for soft-bodied robots. This paper contributes to the state-of-the-art by designing a testbed that features a tendon-driven soft-bodied robot with integrated closed-loop force control.

Deep reinforcement learning (DRL) has emerged as a promising solution to mastering explosive and versatile quadrupedal jumping skills. However, current DRL-based frameworks usually rely on pre-existing reference trajectories obtained by capturing animal motions or transferring experience from existing controllers. This work aims to prove that learning dynamic jumping is possible without relying on imitating a reference trajectory by leveraging a curriculum design. Starting from a vertical in-place jump, we generalize the learned policy to forward and diagonal jumps and, finally, we learn to jump across obstacles. Conditioned on the desired landing location, orientation, and obstacle dimensions, the proposed approach yields a wide range of omnidirectional jumping motions in real-world experiments. In particular, we achieve a 90 cm forward jump, exceeding all previous records for similar robots. Additionally, the robot can reliably execute continuous jumping on soft grassy grounds, which is especially remarkable as such conditions were not included in the training stage.

Using the language of dynamical systems, Imitation learning (IL) provides an intuitive and effective way of teaching stable task-space motions to robots with goal convergence. Yet, IL techniques are affected by serious limitations when it comes to ensuring safety and fulfillment of physical constraints. With this work, we solve this challenge via TamedPUMA, an IL algorithm augmented with a recent development in motion generation called geometric fabrics. As both the IL policy and geometric fabrics describe motions as artificial second-order dynamical systems, we propose two variations where IL provides a navigation policy for geometric fabrics. The result is a stable imitation learning strategy within which we can seamlessly blend geometrical constraints like collision avoidance and joint limits. Beyond providing a theoretical analysis, we demonstrate TamedPUMA with simulated and real-world tasks, including a 7-DoF manipulator.

Soft robots promise inherent safety via their material compliance for seamless interactions with humans or delicate environments. Yet, their development is challenging because it requires integrating materials, geometry, actuation, and autonomy into complex mechatronic systems. Despite progress, the field struggles to balance task-specific performance with broader factors like durability and manufacturability—a difficulty that we find is compounded by traditional sequential design processes with their lack of feedback loops. In this perspective, we review emerging co-design approaches that simultaneously optimize the body and brain, enabling the discovery of unconventional designs highly tailored to the given tasks. We then identify three key shortcomings that limit the broader adoption of such co-design methods within the soft robotics domain. First, many rely on simulation-based evaluations focusing on a single metric, while real-world designs must satisfy diverse criteria. Second, current methods emphasize computational modeling without ensuring feasible realization, risking sim-to-real performance gaps. Third, high computational demands limit the exploration of the complete design space. Finally, we propose a holistic co-design framework that addresses these challenges by incorporating a broader range of design values, integrating real-world prototyping to refine evaluations, and boosting efficiency through surrogate metrics and model-based control strategies. This holistic framework, by simultaneously optimizing functionality, durability, and manufacturability, has the potential to enhance reliability and foster broader acceptance of soft robotics, transforming human-robot interactions.

Generating precise motions with continuum soft robots calls for ways of closing the loop through nonconventional sensors - like cameras. This paper considers, for the first time, model-based visual servoing control of tendon-driven continuum soft robots. We take into account both regulation and trajectory tracking. We especially focus on a system inspired by the human neck, which employs an eye-in-hand configuration. The considered control architecture maps visual feedback to motor commands using the overall system's Jacobian, aiming for accurate end-effector (i.e., robot's head) positioning. This is made possible by blending classic results in visual serving with reduced-order models for continuum soft robots. In this work, we place significant focus on evaluating the controller's effectiveness on the physical prototype, for which we develop a tailored testing setup, which is a novel contribution of this work. Central to this setup is a tendon-driven soft robotic neck. We extensively characterize the control algorithm's performance for several control gains and different operational scenarios. We show that incorporating feedforward velocity estimation into the controller consistently improves performance in trajectory tracking tasks.

Learning from Interactive Demonstrations has revolutionized the way nonexpert humans teach robots. It is enough to kinesthetically move the robot around to teach pick-and-place, dressing, or cleaning policies. However, the main challenge is correctly generalizing to novel situations, e.g., different surfaces to clean or different arm postures to dress. This article proposes a novel task parameterization and generalization to transport the original robot policy, i.e., position, velocity, orientation, and stiffness. Unlike the state of the art, only a set of keypoints is tracked during the demonstration and the execution, e.g., a point cloud of the surface to clean. We then propose to fit a nonlinear transformation that would deform the space and then the original policy using the paired source and target point sets. The use of function approximators like Gaussian Processes allows us to generalize, or transport, the policy from every space location while estimating the uncertainty of the resulting policy due to the limited task keypoints and the reduced number of demonstrations. We compare the algorithm’s performance with state-of-the-art task parameterization alternatives and analyze the effect of different function approximators. We also validated the algorithm on robot manipulation tasks, i.e., different posture arm dressing, different location product reshelving, and different shape surface cleaning.

Quadrupedal jumping has been intensively investigated in recent years. Still, realizing controlled jumping with soft landings remains an open challenge due to the complexity of the jump dynamics and the need to perform complex computations during the short time. This work tackles this challenge by leveraging trajectory optimization and behavior cloning. We generate an optimal jumping motion by utilizing dual-layered coarse-to-refine trajectory optimization. We combine this with a variable impedance control approach to achieve soft landing. Finally, we distill this computationally heavy jumping and landing policy into an efficient neural network via behavior cloning. Extensive simulation experiments demonstrate that, compared to classic model predictive control, the variable impedance control ensures compliance and reduces the stress on the motors during the landing phase. Furthermore, the neural network can reproduce jumping and landing behavior, achieving at least a 97.4% success rate. Hardware experiments confirm the findings, showcasing explosive jumping with soft landings and on-the-fly evaluation of the control actions.

Obtaining dynamic models of continuum soft robots is central to the analysis and control of soft robots, and researchers have devoted much attention to the challenge of proposing both data-driven and first-principle solutions. Both avenues have, however, shown their limitations; the former lacks structure and performs poorly outside training data, while the latter requires significant simplifications and extensive expert knowledge to be used in practice. This paper introduces a streamlined method for learning low-dimensional, physicsbased models that are both accurate and easy to interpret. We start with an algorithm that uses image data (i.e., shape evolutions) to determine the minimal necessary segments for describing a soft robot's movement. Following this, we apply a dynamic regression and strain sparsification algorithm to identify relevant strains and define the model's dynamics. We validate our approach through simulations with various planar soft manipulators, comparing its performance against other learning strategies, showing that our models are both computationally efficient and 25x more accurate on out-of-training distribution inputs. Finally, we demonstrate that thanks to the capability of the method of generating physically compatible models, the learned models can be straightforwardly combined with model-based control policies.

Motivated by the need for efficiency and robustness in repetitive robotic tasks such as locomotion, this study introduces the concept of Natural Motion Manifolds (NMMs) and presents a control method to stabilize and excite motions based on these structures. By considering the intersection of a Poincaré section with a surface comprising a continuum of autonomous evolutions, the proposed controller extends the linearized Poincaré map control from a single orbit to a family of orbits. This allows us to derive simple controllers to excite intrinsic nonlinear resonances and exploit the natural dynamics when varying the energy target (or the running velocity). We validated the method through simulations and experiments on a serial elastic quadruped. Relying on natural dynamics and minimal motor commands, we could implement a bounding gait at desired velocities without needing dynamic compensations. The experiments provide a thorough validation of the feasibility and the benefits of controlled, predictable, and purposeful oscillatory behavior via explicit excitation of a quadruped’s natural dynamics.

Dynamics-based control offers a promising approach to exploring the motion potential of soft robots. However, inherently infinite degrees of freedom of these systems pose significant challenges for dynamics modeling, closely followed by the pressing robustness concerns arising from finite-dimensional approximations. This paper addresses these issues by proposing a physics-informed dynamics learning neural network and an adaptive fractional-order control for continuum soft robots. Specifically, a deep Lagrangian neural network is first developed with an embedded self-attention mechanism to enhance learning efficiency, accuracy, and data sensitivity. Subsequently, an adaptive fractional-order sliding mode controller is designed, leveraging the inherent historical memory properties of fractional calculus. This controller not only ensures robust shape control but also improves response speed and tracking accuracy. To further handle model discrepancies in the learned dynamics and external disturbances, a nonlinear disturbance observer is introduced to effectively estimate and compensate for lumped uncertainties, thereby ensuring reliable performance. Theoretical analysis confirms the closed-loop stability, while both simulation and experiment results validate the high dynamics fitting accuracy of the proposed network, as well as the robust and precise tracking capability of the fractional-order controller.

The existing model-based control strategies for tendon-driven continuum soft robots neglect the dynamics of the actuation system. Nevertheless, such dynamics have an important impact on the closed-loop performance. This work analyzes the influence of the actuation dynamics in tendon-driven continuum soft robots performing trajectory-tracking tasks. To this end, we use singular perturbation (SP) theory to design controllers that account for such dynamics. We provide the analytical formulation of the SP controllers and their in-depth experimental validation. Additionally, we use high-and low-stiffness tendons to experimentally compare the performance of the proposed SP controllers against traditional feedback control schemes that disregard the actuation dynamics. The experimental results show that SP controllers outperform the approaches that neglect the actuation dynamics by reducing oscillations and achieving lower errors without relying on high gains. Furthermore, it is shown that neglecting the actuation dynamics may lead to instability when the tendons have a low stiffness coefficient.

Motor imagery (MI), an important category in electroencephalogram (EEG) research, often intersects with scenarios demanding low energy consumption, such as portable medical devices and isolated environment operations. Traditional deep learning (DL) algorithms, despite their effectiveness, are characterized by significant computational demands accompanied by high energy usage. As an alternative, spiking neural networks (SNNs), inspired by the biological functions of the brain, emerge as a promising energy-efficient solution. However, SNNs typically exhibit lower accuracy than their counterpart convolutional neural networks (CNNs). Although attention mechanisms successfully increase network accuracy by focusing on relevant features, their integration in the SNN framework remains an open question. In this work, we combine the SNN and the attention mechanisms for the EEG classification, aiming to improve precision and reduce energy consumption. To this end, we first propose a noniterative leaky integrate-and-fire (NiLIF) neuron model, overcoming the gradient issues in traditional SNNs that use iterative LIF neurons for long time steps. Then, we introduce the sequence-based attention mechanisms to refine the feature map. We evaluated the proposed noniterative SNN with attention (NiSNN-A) model on two MI EEG datasets, OpenBMI and BCIC IV 2a. Experimental results demonstrate that: 1) our model outperforms other SNN models by achieving higher accuracy and 2) our model increases energy efficiency compared with the counterpart CNN models (i.e., by 2.13 times) while maintaining comparable accuracy.

Planning methods often struggle with computational intractability when solving task-level problems in large-scale environments. This work explores how the commonsense knowledge encoded in Large Language Models (LLMs) can be leveraged to enhance planning techniques for such complex scenarios. Specifically, we propose an approach that uses LLMs to efficiently prune irrelevant components from the planning problem's state space, thereby substantially reducing its complexity. We demonstrate the efficacy of our system through extensive experiments in a household simulation environment as well as real-world validation on a 7-DoF manipulator (video: https://youtu.be/6ro2UOtOQS4).