Rising environmental pressures in aquatic ecosystems due to climate change require robotic systems capable of safe and non-invasive operation. Conventional underwater robots are typically rigid and rely on noisy, high-power actuators, limiting their suitability for sensitive environments such as coral reefs, seagrass meadows, and freshwater lakes and rivers. Here, we present an octopus-inspired robot integrating a hybrid rigid-soft body with compliant tentacle actuation and closed-loop navigation control. Central to the design is an experimental characterization of the actuator force–angle relationship, which enables a model-based feedforward strategy that exploits a locally linear operating regime, avoiding the computational burden of full nonlinear modeling. This feedforward component is combined with proportional–derivative (PD) feedback control to reject disturbances and compensate for model mismatch. The robot achieves an average forward velocity of 0.072 m/s under open-loop operation. Turning experiments show that maneuverability is governed by torque generation, achieving a minimum turning radius of 0.221 m using two-arm actuation. Closed-loop target tracking demonstrates robust navigation under disturbances through integrated vision- and inertial-based state estimation and task-level actuation allocation. Together, these results demonstrate that bio-inspired morphology combined with experimentally grounded hybrid control can yield efficient, adaptive platforms for underwater operation in ecologically sensitive environments.
... Read more

While Diffusion Policy has emerged as a powerful framework for robotic manipulation due to its expressiveness in modeling complex action distributions, its deployment is heavily constrained by high demonstration collection costs. This study presents a systematic empirical investigation into whether joint-training with visual auxiliary tasks can enhance the sample efficiency of diffusion policies under single-task spatial generalization (i.e., variations in object orientations and initial locations). Restricting observation inputs to raw 2D images and low-dimensional robot proprioception, we incorporate four candidate auxiliary tasks: image reconstruction, active object mask extraction, keypoint prediction, and optical flow estimation. We evaluate them with a joint-training framework across two simulated manipulation tasks and one real-world robotic task, using varying amounts of demonstration data. Our empirical findings demonstrate that joint-training with auxiliary tasks indeed provides sample efficiency benefits, particularly in intermediate data regimes. However, we observe that in certain cases, optimization conflicts and gradient interference between auxiliary and primary tasks diminish these benefits, especially in data-starved or data-rich regimes under simulated settings. ... Read more

Continuum soft manipulators (CSMs) offer key advantages over rigid robots for on-orbit servicing (OOS), but their deployment requires realistic ground-based testing environments. To emulate microgravity conditions for CSMs, this paper proposes tendon antagonism as a model based gravity compensation method. It is shown that a tendon layout design consisting of one linear tendon and two tendons shaped as five-segment piecewise-linear approximations of sinusoids is sufficient to counteract planar bending moment by gravitational loading. Dynamic simulations compare a gravity-loaded plant model, a gravity free reference model, and an uncompensated baseline using a geometric variable strain formulation. Results show accurate tracking on bending DOFs with negligible (10−5 rad/m) tracking errors in steady-state conditions and remain small during motion (10−2 rad/m), increasing moderately at higher actuation levels. Small errors in task space (18 mm) stem from uncontrolled elongation DOF, as tendon actuation is limited to tension only. The design is further validated experimentally, using machine vision for pose estimation. Results show high accuracy of the simulation, and demonstrate that tendon antagonism reduces gravitational sagging and enables tracking of the reference configuration across actuation values. Average positional error reduces by 80%, from 103 mm to 18 mm when using the compensation mechanism. At high actuation values, results show overcompensation for the gravity load, hinting to possible improvements in the tendon control scheme. Future research could implement real-time control or control of the remaining free DOFs. ... Read more

Vision-based robotic grasping has become a widely adopted approach in both industrial and domestic settings, enabling robots to perceive and interact with the environments. While progresses have been made in grasping static or predictable objects, dynamic object grasping remains a challenging problem due to the need for reactive perception, motion prediction, and real-time control. Traditional systems often rely on conventional RGB cameras, which have low temporal resolution and motion blur, resulting in perception latency or missed fast movement. Moreover, many of the previous grasping approaches require precise object or robot models but they are either difficult to obtain or fail in unstructured or fast-changing environments. To address these challenges, Ev2Act-Grasp is proposed, an end- to-end event vision-based deep reinforcement learning grasping system, which directly maps event frames inputs to 3D Cartesian control actions, performing a tracking–grasping task. The system operates in a fully end-to-end manner—without prior object models, handcrafted features, or supervised perception modules—using a flexible eye-in-hand configuration to handle randomly moving spheres. Ev2Act-Grasp is evaluated in simulation using a Franka Panda robot and demonstrate that it achieves a 100% tracking success rate and a 66% grasping success rate in the clean scenario, while maintaining over 75% tracking success
in moderately cluttered environments. Furthermore, the system demonstrates zero-shot sim-to-real transfer, achieving successful grasping across clean, clustered, and low-light environments under various object motions. ... Read more

Continuum soft robots present significant opportunities for advancing robotics, but they also introduce substantial technical challenges. These systems are highly nonlinear, infinite-dimensional, and severely underactuated, making control particularly difficult. While recent advancements in model-based control have addressed some of these issues for soft robotics, numerical optimal control has shown strong potential, especially given its success in other severely underactuated domains such as bipedal and quadrupedal locomotion.

However, the application of optimal control in soft robotics has largely relied on simplified models, and its use with more accurate and geometrically consistent formulations remains underexplored, particularly for explicitly tackling underactuation. This thesis investigates the use of Differential Dynamic Programming (DDP) to control continuum soft robots modeled using the Geometric Variable Strain (GVS) framework. The focus is on the Soft Inverted Pendulum (SIP) as a template system to evaluate DDP’s feasibility, robustness, and performance in underactuated settings, including low-stiffness regimes where collocated feedback strategies break down. The implementation leverages the use of analytical gradients computed via the Recursive Newton-Euler Algorithm (RNEA) to improve convergence and computational efficiency.

The results show that DDP outperforms traditional Partial Feedback Linearization (PFL) methods, both collocated and non-collocated, especially across challenging mass-stiffness combinations. This effectively extends control authority and stability into regimes previously considered difficult to handle. This thesis extends the method to more complex hybrid soft–rigid systems, examining real-time feasibility and practical implementation, thereby laying the foundation for a generalizable optimal control framework for soft robots. ... Read more

This paper presents a robotic grasping system that integrates soft robotic fingers, a reconfigurable gripper, and a YOLOv11-OBB-based object detection to enable intelligent adaptive, grasping. The system addresses the challenge of handling fragile and geometrically diverse objects common in agricultural and food-handling applications by dynamically adjusting its gripper configuration in response to object characteristics. A novel soft finger, selected through finite element modeling and experimental validation, provides compliant contact. The object detection model not only localizes and orients objects but also infers the optimal finger configuration, encoded via class ID. Experimental results demonstrate significant performance improvements: grasp success rates increased from 70% to 84,67% for fruits and from 66% to 84,67% for abstract objects, with only a modest 3-second increase in cycle time due to reconfiguration. ... Read more

Fully actuated robots may be controlled using well-understood techniques, such as computed torque control, which override natural system dynamics. For underactuated robots these dynamics cannot be fully cancelled out, and must instead be leveraged, complicating the control problem. We present a classification of underactuated robotic systems based on the degree to which their dynamics can be decoupled. Finding coordinates that decouple the system dynamics simplifies control for two classes of robots, which we identify as partially- and fully decouplable robotic systems. In these decoupling coordinates, the Euler-Lagrange system representation has a block-diagonal inertia matrix and decoupled input matrix. After delving into the fundamentals of this proposed classification, this work implements an autoencoder as a first ML-based framework to learn these decoupling coordinates for the 2 degrees of freedom (DOF) case. Furthermore, we demonstrate how such representations simplify control. Input decoupling allows for collocated control using a straightforward PD + gravity compensation controller. Inertial decoupling enables non-collocated control through feedback linearization within a small set of states. To demonstrate the theory, decoupling coordinates are learned for a 2-DOF toy system. Performance of the learned coordinate transform is analysed, and controllers on learned and analytic decoupling coordinates are compared. ... Read more

Through the use of Eigenmanifold theory, unforced periodic trajectories called nonlinear normal modes can be identified and excited in nonlinear mechanical systems. Applying this to repetitive tasks in robotic systems with elastic components can drastically reduce energy consumption, as normal modes in steady-state require no additional control effort. However, existing control methods for exciting nonlinear normal modes have so far only assumed full actuation. Consequently, these techniques are incompatible with series elastic joint robots, even though they represent a significant subclass of physical systems with elastic elements. Additionally, the calculation and parameterization of Eigenmanifolds for high-dimensional systems generally remains a complex task and is difficult to scale. While existing literature aims to avoid forced evolutions or model cancellation, we instead lean into this approach. By rephrasing Eigenmanifold-based control as a trajectory tracking problem, standard techniques for elastic joint robot trajectory tracking control can be employed. Furthermore, obtaining theoretical guarantees on global stability becomes possible. In this work, a new modular control architecture is presented that integrates trajectory tracking feedback control with Eigenmanifold theory to dynamically generate and track hyper energy-efficient oscillatory movements in underactuated systems. This approach enables the excitation of nonlinear normal modes using standard trajectory tracking controllers, while preserving energy-efficient properties desired from Eigenmanifold-based controllers. We first discuss the theoretical validity and energy-efficiency of this control architecture, and then test the architecture in simulation for a variety of use-cases and controllers. ... Read more

Soft continuum robots provide a compelling solution for safe interactions with unknown and unstructured environments, owing to their compliance and infinite degrees of freedom. However, the non-linear deformation and inherent underactuation pose a persistent challenge for real-time control and state estimation. To date, continuum structures are typically discretized using a fixed-parameter kinematic model, introducing a trade-off between model accuracy and computational efficiency.

This letter presents an adaptive kinematic modeling framework to address the challenge of underactuation in a different way - not by increasing model resolution or complexity - but by making the model parametric with respect to a set of parameters that are dynamically adapted using a novel inverse kinematic adaptive controller.

We formally prove stability of the adaptive controller and validate its performance through simulations and experiments, considering setpoint reaching tasks with variable end-effector payloads. Even though the approach introduces additional challenges, in comparison to the conventional fixed-parameter models presented in literature, the proposed solution enhances shape representation, redundancy resolution, and state inference while mitigating model complexity. ... Read more

To make interactions between humans and robots safer, soft robots may offer a solution. The autonomous closed loop control of these robots so far, however, is not accurate enough to perform specific tasks as handovers. The purpose of this paper is to propose a control algorithm that can make the control of soft robots accurate enough for human-to-robot handovers. The main focus within this research was on the state estimation and the Jacobian based control. Due to gravity, the internal system state is not an accurate representation of the actual system behavior. The optimized pose estimation solves this problem. In order to test the proposed algorithm, the complete control architecture has been implemented including the object and end-effector detection. Experiments have shown that the algorithm works with different step sizes within the Jacobian based control,
consistently resulting in a successful handover. A second experiment has shown that the handovers are still successful and faster when a human guides the robot toward the right position. Lastly, a possible use case has been shown. ... Read more

Quadrupedal robots have great potential for deployment in challenging environments. However, one of the most significant challenges these robots face is maintaining stability on slippery surfaces due to inaccurate friction estimation. This thesis investigates the role of accurate friction estimation in improving locomotion and reducing slip in quadrupedal robots. A Model Predictive Controller (MPC) with a friction-aware constraint update is proposed and evaluated in a simulation environment using a physics-based simulation with Open Dynamics Engine (ODE).

The experimental results demonstrate that incorporating real-time friction measurement and constraint in the MPC framework significantly reduces slip occurrences, decreases energy consumption, and improves overall locomotion stability. Statistical hypothesis tests, including paired t-tests and Bonferroni corrections, confirm the significance of these improvements. The findings suggest that integrating real-time friction estimation into quadrupedal robot controllers can enhance their robustness and reduce the need for explicit slip recovery strategies. Future work should focus on extending this approach to real-world scenarios, incorporating actual friction sensors, and testing in diverse terrains. ... Read more

This thesis presents the design and control of the TActile Soft Quadruped (TASQ), a pneumatically actuated soft robot equipped with integrated tactile sensing for adaptive locomotion. Two core contributions are introduced. First, a novel tactile suction cup sensor is developed, capable of simultaneously providing foot contact information and generating suction-based adhesion. The sensor combines embedded magnets and magnetometers to estimate ground reaction forces via a learned calibration model, enabling lightweight, compliant, and robust tactile feedback essential for closed-loop control in soft robotics. Second, a learning-based control framework is proposed that integrates behavior cloning with domain-randomized reinforcement learning to achieve adaptive and robust locomotion. The approach first imitates a reference gait to initialize a stable walking policy and then refines it in simulation using the Soft Actor–Critic algorithm. The learned policy exploits proprioceptive and tactile feedback to enable goal-directed, stable motion and transfers effectively from simulation to real hardware. Experimental validation demonstrates that the learned closed-loop controller outperforms open-loop control on the physical robot, improving forward speed by 41\% on flat terrain and by 91\% on a $2.5^{\circ}$ incline. Ablation studies further confirm the importance of tactile and inertial feedback for stability and performance. Overall, this work establishes a unified sensing and learning framework for a soft legged robot, paving the way toward adaptive, environment-aware locomotion without reliance on vision. ... Read more

Safe quadrupedal locomotion control with reinforcement learning (RL) has attracted increasing attention in recent years, where existing approaches can be broadly categorized into recovery RL, distributional RL, and constrained RL. However, recovery RL cannot provide predictive safety guarantees; distributional RL lacks passive safe performance; and constrained RL-while capable of both safety-often restricts exploration. To address these limitations, we propose \textbf{UPPS-RL}, a unified framework that integrates predictive and passive safety into quadrupedal locomotion control through three main components: a risk-aware task-level policy, a self-supervised risk network, and a risk-triggered recovery policy, forming a hierarchical control architecture that embeds unified safety without imposing explicit exploration constraints. Extensive simulations across composite scenarios, including steps, pit, slope, and rough plane terrains, demonstrate that UPPS-RL significantly suppresses catastrophic failures while maintaining a favorable trade-off between robustness and efficiency. ... Read more

Designing robotic systems such as quadrupeds is challenging due to the intricate relationship between motion and design, particularly when aiming to replicate the agility, efficiency, and versatility of animals. Co-design simplifies robotic development by simultaneously optimizing physical design and control algorithms in an integrated way. While most prior work validates co-design approaches in simulation, our research bridges this gap by transitioning optimized designs to real-world implementation. To achieve this, we developed a modular quadruped platform with bio-inspired legs that enables the physical implementation of the optimized designs. Our design space, which includes leg segment lengths, spring stiffness, and engagement angle, was optimized to maximize energy efficiency for real-world tasks. We propose a simplified learning-based co-design framework that combines reinforcement learning to create a universal locomotion controller with Bayesian optimization to select the best design. Real-world tests demonstrate a significant reduction in the cost of transport—18.6% for inspection tasks and 35.7% for payload tasks—compared to the nominal design without springs. In simulations, the universal controller adapts well across robot configurations, and the optimization process remains consistent across runs. Although some discrepancies between simulation and real-world performance remain, our findings underscore the potential of co-design to address complex trade-offs in real-world robotic system design. ... Read more

As we increasingly strive to integrate robots into human-centric environments, safety is a top priority. Traditionally, rigid collaborative robots have relied on safety-aware computational control policies, which are susceptible to perception errors and often lead to overly cautious behavior that limits performance. In contrast, soft robotics offers a promising alternative by ensuring passive compliance throughout the robot’s structure via material softness. This mechanical compliance inherently mitigates safety issues arising from perception or control errors, although this has been paid with a substantial drop in precision. In recent years, significant advances have been seen in soft robotics, with exciting new developments in design, smart materials, actuators, sensors, models, and control strategies. However, the modeling and control of continuum soft robots continue to pose major challenges due to their infinite degrees of freedom, complex nonlinear dynamics, and time-dependent behaviors like hysteresis. As a result, soft robots often lack the necessary capability and motion precision, leading to a tradeoff where performance is sacrificed for safety. With this thesis, we argue that this tradeoff can be overcome by developing more advanced algorithms that can reason on the physics of the soft robot. More specifically, we propose combining powerful learned models with efficient and effective model-based control approaches that allow for interpretability into the actions and admit stability guarantees.

Currently, two main approaches exist for controlling soft robots. The first employs model-based control using approximated physics-based models derived from first principles. The second directly learns control policies, primarily through reinforcement learning. Both strategies face notable limitations. Existing model-based controllers are unable to fully manage and eventually exploit the dynamics of soft robots because their underlying models inadequately capture complex behaviors, particularly how actuation and external interactions affect the robot’s deformation. Moreover, deriving these models requires extensive expert knowledge. Additionally, the combined complexity and uncertainty of the dynamics between a soft robot and its environment make it currently infeasible to develop comprehensive world models from first principles alone, thereby motivating the integration of machine learning approaches that can effectively leverage data-driven insights. Conversely, directly learning the controller — such as via reinforcement learning — lacks interpretability and stability guarantees while being highly sample inefficient, a significant drawback given the time-dependent material properties and limited lifespan of current soft robots.

In this thesis, we contend that combining learned models with model-based controllers presents a promising alternative that brings together the advantages of both approaches: expressive, data-driven models that require less expert knowledge paired with controllers that are both interpretable and provably stable. Although recent years have seen increased interest in leveraging learned models for control, most work in this area depends on computationally intensive optimal control methods, such as MPC, to optimize the actuation sequence with the learned model. However, the high computational cost of solving these optimal control problems limits the maximum control frequency during deployment, preventing us from fully exploiting the dynamic capabilities of soft robots. Instead, this thesis pursues closed-form controllers that utilize the physical structure of learned models within an energy-shaping framework. The main challenge here is to develop approaches that integrate such physical structures—specifically, kinetic and potential energy terms—into the learning of dynamical models for soft robots. Before addressing this main challenge, we first had to advance physics-based models derived from first principles and identify novel techniques to leverage them for control. On one hand, this clarified which physical priors were available for learning, while on the other hand, it inspired new ways to integrate model-based controllers with learned models. The thesis addresses this topic through several interconnected key contributions.

First, we argue that quantifying the safety of soft robots is crucial for designing and controlling them to ensure that the closed-loop system meets the specific safety requirements of their intended applications. To this end, we present the first safety metric for continuum soft robots, which assesses the safety of an integrated soft robot design by accounting for both its embodied and computational intelligence.

Secondly, this thesis enhances shape sensing for soft robots by leveraging insights from kinematic models. We accomplish this by formulating and solving nonlinear optimization problems that align sensor measurements with the backbone shapes predicted by the kinematic model. We present two distinct approaches that integrate commercial sensors—namely visual and magnetic—with SLAM algorithms and a learned sensor measurement model, respectively, to accurately estimate the soft robot’s state, a key requirement for effective feedback control.

Thirdly, this thesis introduces advanced physics-based actuation models, including those for robots actuated by auxetic metamaterials - referred to as HSA robots—and models that capture the actuation dynamics of piston-driven pneumatic soft robots. We then leverage the acquired model insights to design provably stable nonlinear controllers—specifically, an integral-saturated PID combined with potential shaping and Cartesian-space impedance control for planar GSA robots, as well as a backstepping controller for pneumatic piston-driven soft robots. This contribution deepens our understanding of actuation, a critical aspect of soft robot behavior, and demonstrates how such insights can be incorporated into model-based control strategies. Moreover, experiments with HSA robots have highlighted the limitations of purely physics-based models in capturing complex phenomena like hysteresis, thereby motivating the exploration of learning-based approaches. In the future, the developed actuation models can serve as valuable physical priors for learned models.

Fourthly, the thesis presents techniques for learning soft robot models that incorporate physical structures while ensuring stability. We accomplish this by embedding physics-based dynamical models into the learning algorithm, which determines the free parameters of the dynamics and optionally optimizes a coordinate transformation—such as encoding into latent space. Two notable approaches are introduced: (1) an algorithm that extracts low-dimensional soft robot strain models from samples of the robot backbone’s shape evolution, and (2) a network of coupled harmonic oscillators for learning latent dynamics from high-dimensional observations like images. The explicit inclusion of kinematic and potential energy terms in these models allows for stability analysis using standard nonlinear system theory tools, such as Lyapunov methods. For instance, we prove that the coupled oscillator network is both globally asymptotically stable and input-to-state stable.

Fifthly, we exploit the physical structure of the learned models from contribution four to design closed-form setpoint regulators. The controller contains two key components: (1) a potential shaping feedforward term that positions the local/global minimum of the closed-loop potential energy at the setpoint by leveraging the learned model knowledge, and (2) an integral-saturated PID feedback term that rejects disturbances and compensates for modeling errors to prevent steady-state errors. The stability of the closed-loop system can then be analyzed using Lyapunov arguments.

Finally, the thesis explores methods for generating compliant motion behaviors in soft robots beyond low-level control. One approach focuses on assisting users, particularly elderly individuals, with activities of daily living by guiding the low-level controller with brain signals. This is achieved by combining motor imagery classification from wearable EEG devices with compliant impedance control in operational space. The second approach combines an orbitally stable dynamical system in latent space with a bijective neural network parametrized encoder to learn periodic motions from demonstrations. By avoiding reliance on time references, this learned motion policy enables natural and compliant tracking of demonstrated periodic motions. This contribution ensures that not just the robot structure and low-level controller are compliant, but also the high-level motion strategy. ... Read more

Reinforcement learning (RL) is a powerful tool where the agents – or “robots” can learn from the environment based on their actions. Reinforcement learning approaches were found successful in combining predicting stock returns and portfolio allocation. Diversification is a critical element for achieving high portfolio returns with a lower level of risk. This work explores the application of reinforcement learning and multi-agent reinforcement learning (MARL) in portfolio management, emphasizing the applicability and increased portfolio performance via strategy diversification. A key contribution of this work is the development and evaluation of a MARL environment incorporating novel diversity measures, correlation, and total variation distance. The findings reveal that while fine-tuned single-agent RL models can demonstrate strong performance, roughly tuned MARL models with diverse agents reflecting a "portfolio of portfolios" paradigm show improved action diversification and portfolio performances. The work also highlights the critical role of long-term robustness testing, algorithm- and problem-specific hyperparameter optimization, and the challenges of adapting MARL methods to financial contexts. This work contributes to the RL research in portfolio management by exploring the use of MARL in portfolio management and discussing the limitations and future work directions. ... Read more

The rapidly growing volume of parcel shipments is straining transportation and logistics sectors, highlighting the need for innovative solutions to optimize packing and loading processes. The online bin packing problem (BPP), an NP-hard computational problem, finds practical applications in numerous sectors, including modern packaging and intelligent logistics. This study proposes a novel reinforcement learning (RL) approach to tackle the online 3D-BPP emphasizing applicability and versatility. The key innovation is the representation of the packing scene as a graph, enabling effective encoding of task-specific high-level features. This graph-based structure serves as the foundation for an RL agent designed to learn an optimal packing strategy through dynamic interaction with the environment. The proposed approach uniquely operates within the continuous domain, enhancing generalization across diverse packing tasks. Experimental evaluations in both simulated environments and a real-world setting demonstrate that the solution achieves state-of-the-art performance across multiple complex three-dimensional packing scenarios. ... Read more

This work introduces a novel methodology for the development of interpretable reduced-order dynamic models specifically tailored for jumping quadruped robots. Leveraging Symbolic Regression combined with autoencoder neural networks, the framework autonomously derives symbolic equations from data and fundamental physics principles capturing the complex dynamics of jumping actions with high fidelity. This approach significantly reduces model complexity while enhancing interpretability, facilitating deeper insights for legged robotic applications. The efficacy and accuracy of the proposed models are validated through comprehensive experimental studies, marking a substantial advancement in the design of agile and efficient legged robots. This research demonstrates the outperformance of a learned 2D model compared to existing template models such as the ASLIP. Also, an analysis of the dimensionality of the learned model is conducted showing the overarching tradeoff between accuracy and complexity. The method is validated on different simulated quadrupeds and an actual hardware robot. ... Read more

With the current transition towards renewable and high-tech solutions, the world is becoming increasingly complex. Consequently, the challenges faced by firefighters also intensify. For that reason, firefighting robots are rising in popularity despite being far from perfect. An important area of improvement is the perception capabilities of those robots, given the fact that firefighting robots suffer from occluded camera views in environments filled with smoke. To overcome this challenge a LiDAR sensor may be used but experiments in this work show that even those point clouds are adversely affected by smoke. Consequently, this work presents a method for real-time reconstruction of ground surfaces in occluded environments filled with smoke. The developed method functions in ROS Noetic and merges segmented ground points, when available, with ground surfaces which are reconstructed based on information from segmented wall points. In this way, the method works even without the presence of ground points. To achieve this, a combination of established techniques from scientific literature, along with newly developed techniques were implemented. Doing so gives the robot’s operator an improved representation of the ground surface within environments filled with smoke. Ultimately the developed method may allow for autonomous navigation based on LiDAR data within environments filled with smoke. This research shows that a method consisting of techniques which tackle the independent sub-challenges arising from the use of LiDAR in indoor environments filled with smoke can effectively reconstruct the ground surfaces within those environments. Furthermore, the developed method has the potential to do so in a real-time manner.
... Read more

The generation of a 3D map of an unseen environment, obtained through solving the SLAM problem, is a popular topic currently in the robotics domain. The Lunar Rover Mini (LRM) at the German Aerospace Center solves this problem using a RGB-D camera system, which is favourable in space applications due to its lightweight characteristics and energy-efficiency. Performing SLAM based on camera images is based on visual odometry: the science of estimating the rover’s trajectory trough a sequence of images. However, the dependency on a
single sensor to perform mapping and navigation poses a threat to the reliability of the system. To increase the reliability and robustness of the SLAM algorithm, an inertial measurement unit (IMU) is incorporated in the robot hardware.
This thesis describes the design for a visual-inertial SLAM algorithm that incorporates both visual and inertial measurements to solve the SLAM problem through performing tightly coupled sensor fusion, which estimates and corrects for IMU biases. The solution is based on a non-linear factor graph, which is a graphical model to represent the relationships between the
rover’s measurements and the unknown variables which are optimised for. This is done using the open-source GTSAM framework. Using experimental data, the robustness of the novel visual-inertial SLAM algorithm is demonstrated by simulating specific sensor failures. Moreover, the novel algorithm shows its capability to incorporate a degree of certainty regarding specific areas of the generated map, closely resembling how a human being would generate a
map of an unknown area.
An additional use case for tightly coupled sensor fusion is the increased accuracy of the estimated trajectory. Assuming Gaussian noise models for both measurement models, averaging the two can yield a higher accuracy than either of the two sensors could have obtained by itself. This hypothesis was tested in another experiment. As the main mechanism behind bias estimation is reducing the error between visual and inertial measurements, bias estimation is quickly affected by this drifting visual odometry, which in its turn deteriorates the accuracy of the visual-inertial odometry module. This observation proves that the bias estimation is not correlated to the underlying physical process, but is rather just a numerical value in the optimisation reducing the residual error. It raises the question whether this strategy of tightly coupled sensor fusion can actually be used to increase the accuracy of a visual odometry algorithm. ... Read more