Explicit trajectory communication can be used to coordinate multiple robots, but communicating at every planning iteration can lead to congestion of the communication network, increase message delays and message loss. At the same time, collision-free trajectory planning is often formulated as a nonconvex optimization problem, which can converge to different locally optimal solutions across consecutive planning iterations.When this happens, a robots planned motion can switch between distinct high-level avoidance behaviors, such as passing an obstacle on the left versus on the right, which can lead to inefficient or unsafe behavior. Topology-based motion planners address this by explicitly computing multiple candidate motion plans that represent these different passing decisions, each associated with a distinct homotopy class. This work asks how (changes in) homotopy-class representations can trigger communication to reduce communication load while maintaining safe and efficient behavior. Building on the topology-driven trajectory optimization(T-MPC) approach of [ 1], we propose T-DMPC, a topology-aware distributed motion planner in which each robot, computes multiple guidance trajectories in distinct homotopy classes and refines them via parallel local trajectory optimization within the corresponding homotopy classes, selects a solution using a consistent decision rule that prioritizes the previously executed homotopy class, and communicates the selected trajectory using an event-triggered policy. Communication is triggered by homotopy changes and complemented by geometric-deviation and time-based triggers to bound trajectory staleness. In addition, the robots communicate a fallback trajectory during planning failures (e.g. infeasibility). We evaluate T-DMPC on antipodal swap maneuvers with 2 and3 robots in simulation and on physical robots, comparing against T-VMPC (no communication, constant-velocity predictions) and T-AMPC (always communicate). The experiments show that, T-DMPC achieves task duration and traveled distance comparable to T-AMPC and T-VMPC, while reducing communication to about 9.8% (2 robots) and 13.2% (3 robots) of planning iterations in simulation, and 17.8% (2 robots) and 13.2% (3 robots) in real-world experiments, with no observed physical collisions. Ablations however show that topology-change alone is insufficient for safety,motivating the combined trigger design.

Autonomous driving near the handling limits of a vehicle places stringent demands on the control layer, where nonlinear dynamics, actuator delay, and model mismatch dominate closedloop behavior. Sampling-based optimal control methods, and in particular Model Predictive Path Integral (MPPI), offer an attractive alternative to deterministic receding-horizon control through stochastic rollouts and importance weighting. While MPPI has demonstrated strong performance in simulation, its real-world behavior at high speed remains insufficiently characterized.
This thesis investigates the deployment of baseline MPPI and three smoothness-oriented extensions, Dynamic Covariance MPPI, Smooth MPPI (SMPPI), and Low-pass Filtered Sampling MPPI (LFS-MPPI), on a small-scale autonomous racing platform. Experiments focus on high-speed trajectory tracking under tight lane constraints and obstacle interactions, where steering bandwidth limitations and actuation delay significantly influence stability. Results show that baseline MPPI achieves competitive speeds and low nominal tracking error, but generates excessive high-frequency steering activity that induces sustained oscillations on hardware. Stability improvements are consistently associated with reduced steering-rate excitation. Among the evaluated extensions, LFS-MPPI provides the most favorable tradeoff between smoothness, tracking accuracy, and robustness by shaping the sampling process rather than post-processing the control signal.
The experiments further reveal a fundamental coupling between model fidelity and sampling efficiency under strict real-time constraints. By increasing the proportion of dynamically plausible rollouts at reduced sample counts, LFS-MPPI enables stable deployment of a dynamic bicycle prediction model and reliable tracking up to 3.5 m/s on the physical racetrack. These findings demonstrate that sampling structure is a decisive factor in achieving stable, high-performance, real-world MPPI control.

Autonomous perching enables micro aerial vehicles (MAVs) to act as quiet, non-intrusive sensing nodes for ecological monitoring and habitat assessment. We present a 1.2[kg] quadrotor designed for selective branch perching in natural environments, aiming to extend operational endurance and enable elevated observation in dense vegetation.
We achieve autonomous quadrotor perching by addressing challenges in perception, planning, mechanical design and control.
Our system, equipped with a stereo vision pipeline and a lightweight top-mounted actuated gripper arm, demonstrates fully autonomous perching across a wide range of branch and stem orientations - from horizontal to vertical - even in cluttered, near-natural forest scenes. In controlled laboratory trials, the MAV performs a 360° scan to identify candidate perches, verifies them with depth-based checks, executes a bottom-up, orthogonal approach, and supports repeated take-off and re-perching at different branches.
This is the first time the complete pipeline of design, perception, planning, and control is addressed for perching drones. With that, our work lays the foundation for future deployment in ecological monitoring and restoration missions, where minimally invasive aerial observation could benefit remote or sensitive ecosystems.

As self-driving vehicles progress toward real-world deployment, efficient and reliable motion planning in dynamic multi-agent environments becomes increasingly essential. This work addresses this challenge by advancing the field of nonlinear distributed model predictive control (NMPC) for autonomous multi-vehicle coordination. The approach focuses on complex, interaction-dense scenarios where space is limited. To model vehicle shapes and collision boundaries efficiently it uses polytopic set representations to approximate vehicle geometries and collision boundaries instead of conventional shapes such as ellipsoids.

We propose a novel distributed NMPC approach for navigation in tight environments. The goal is to enable coordinated motion planning for multiple autonomous vehicles in dense traffic scenarios. This can be easily modelled with centralised formulations, however, considering the scale of the road network they become computationally intractable as the number of agents grows. In distributed approaches instead each vehicle solves its local part of the problem which scales linearly and is therefore better suited for these kind of environments. So building on an existing distributed method we enhance its structure to improve scalability, safety, and performance in cooperative autonomous driving tasks. The research is guided by three objectives: (RO1) to reproduce a known distributed baseline algorithm using the DART (Delft’s Autonomous-driving Robotic Testbed) vehicle model \cite{lyons_dart_2024}, the Model Predictive Contouring Control (MPCC) \cite{lam_model_2010} tracking objective, and the ACADOS solver framework \cite{verschueren_acados_2020}; (RO2) to develop a novel distributed MPC-based algorithm that improves inter-vehicle spacing and tracking performance while generalizing beyond predefined reference trajectories; and (RO3) to prepare the algorithm for validation on a real-world robotic testbed.

The primary contributions of this work include: (C1) successful reproduction of the distributed baseline using open-source tools and realistic vehicle modelling; (C2) development of two enhanced distributed algorithms, Distributed Model Predictive Contouring Control with Relaxed Collision Avoidance (DMPC-RCA) and DMPC-RCA with consensus (DMPC-RCA-C), that demonstrate superior performance in tracking accuracy and safety margins; and (C3) integration of these algorithms with the DART hardware platform for future experimental validation. The repository of these approaches can be found in https://github.com/HubertVisser/multi-vehicle-coordination-algorithm.git

Performance is evaluated in two representative scenarios, a merging situation and a T-junction, with the centralised NMPC approach serving as the performance benchmark. To evaluate the performance of the proposed designs, three key metrics are used: accumulated tracking cost, computation time, and minimum inter-agent distance. The results show that both proposed methods achieve tracking costs comparable to the centralised controller, while significantly outperforming the distributed baseline method. Notably, the inclusion of a consensus term yields no substantial improvement in performance over the non-consensus version.

To conclude, the proposed approaches offer strong potential for scalable, safe, and efficient multi-agent motion planning, moving one step closer to the deployment of fully cooperative autonomous driving on public roads.

Github repository for the Master Thesis - https://github.com/HubertVisser/multi-vehicle-coordination-algorithm.git

This thesis proposes a study towards the application of imitation learning (IL) algorithms Action Chunking Transformer (ACT) and diffusion policy as an autonomous surface vessel (ASV) path planner in complex marine environments. Rationale for conduct- ing this research are the ubiquitous limitations regarding fine tuning cost- and or reward functions of conventional state of the art algorithms for ASV navigation in complex environments. In this study we trained both algorithm types on data sources collected from a basic 2D-grid sim- ulator and a more realistic GAZEBO simulator. Subse- quently we evaluated both algorithm’s performances in each respective simulator in terms of the success rate for a standard navigation task. We found relatively high suc- cess rates for the 2D-grid simulator (0.98 for ACT and 0.53 for diffusion policy). For the GAZEBO simulator, we found poor performance for ACT (0.0 success rate) and for diffusion policy we could not establish the perfor- mance due to hardware limitations. For future work, the capabilities of both models could be investigated further by trying to bridge the gap between the simple 2D simula- tor and the GAZEBO simulator. Mainly the effect of task complexity and the quality and quantity of data used for training the models on the performances of the models can be investigated in these future work studies.

Plastic pollution in rivers is a growing environmental issue with widespread impacts. Monitoring the movement of plastic waste across different river systems is challenging due to environmental variability and the limited availability of labeled data. This thesis investigates camera-based methods for detecting floating macroplastics in rivers and explores ways to adapt detection systems to new locations with minimal data. Collecting data from the Limmat River in Zurich and the uMhlangane River in Durban, South Africa, the study assesses the impact of domain shifts on detection performance and proposes a semi-automated annotation pipeline to improve labeling. Furthermore, it tests techniques like few-shot learning and pseudolabeling to address the performance dip. The results show that model performance decreases significantly when applied to new locations but that even with minimal data, camera-based monitoring can provide useful insights for understanding waste movement and informing plastic waste management strategies.

In this paper, we address the Blocking Flexible Job-Shop Scheduling Problem with Transportation and Time Windows (BFJSPT-TW), which combines assignment, blocking, and transport constraints under spatial feasibility.
Exact monolithic formulations rapidly become intractable as instance size increases. To overcome this, we propose a Logic-Based Benders Decomposition (LBBD) that integrates a disjunctive-graph relaxation into the master problem, providing informative lower bounds and accelerating convergence.
The subproblem is solved through Constraint Programming, ensuring temporal and blocking feasibility.
Computational experiments demonstrate that the proposed LBBD–DG achieves near-optimal solutions with up to 80 % stronger initial bounds and four times faster convergence than baseline models.
The method establishes a scalable near-exact framework for layout-aware scheduling that is capable of producing high quality schedules in minutes for very large problem instances.

The rapid increase in drone threats has created an urgent need for practical counter-drone systems for safety and defence purposes. Interceptor drones represent a promising and cost-effective solution for removing unwanted aerial vehicles. However, classical guidance laws such as Proportional Navigation (PN) are not well adapted to highly agile targets under realistic perception constraints. In parallel, deep reinforcement learning (DRL) has demonstrated exceptional performance in high-speed quadcopter control tasks, motivating its consideration for interception applications. A major challenge for such learning-based controllers lies in sim-to-real transfer. This work addresses this challenge through extensive domain randomization and explicit modelling of realistic perception constraints, including sensor noise, limited field of view, sensing ranges, and sensor update rates. A framework is proposed that trains Single Rotor Thrust (SRT) DRL policies for quadcopters and evaluates them against a classical PN implementation within a physics-based simulator. The resulting controllers generalise across platform variations and outperform classical guidance under modelled perception limitations, demonstrating improved robustness and transferability.

Github Repository of the code used for the thesis: https://github.com/maximecapelle/quadintercept_drl

Offshore geotechnical exploration presents unique challenges due to harsh environments, limited sensor data, and the need for high-fidelity soil characterization. This has led to the deployment of automated solutions, such as subsea platforms, that support exploration activities. These robotic systems are often coupled with intelligent systems capable of performing tasks like drilling autonomously. This work focuses on developing an intelligent advisory system for the Blue Dragon subsea platform to enable autonomous drilling operations. To address the challenges presented by this unique application, the research explores multi-task learning to enhance timeseries forecasting models that predict optimal drilling parameters including thrust, torque, and rotation speed.
Four model architectures were evaluated: a baseline forecasting model, a forecasting model enriched with a separately trained lithology classifier, a jointly trained classifier-forecaster system, and an unsupervised embedding extractor approach. These models were tested on both synthetic vehicle dynamics data and real-world drilling data from Blue Dragon® operations. While the synthetic data demonstrated theoretical soundness of the approaches, evaluation on the drilling dataset revealed significant limitations due to insufficient data quality and quantity, resulting in severe overfitting that prevented conclusive model validation.
The results indicate that although lithology-informed forecasting shows theoretical promise, current data constraints prevent robust generalization to operational conditions. Future work should prioritize acquiring production-grade datasets and developing refined data preprocessing techniques to better isolate drilling-relevant signals. This research establishes foundational methodology for safe, efficient, and adaptive automation in offshore drilling, contributing to the broader objective of autonomous geotechnical exploration.

Reinforcement learning (RL) is a powerful tool where the agents – or “robots” can learn from the environment based on their actions. Reinforcement learning approaches were found successful in combining predicting stock returns and portfolio allocation. Diversification is a critical element for achieving high portfolio returns with a lower level of risk. This work explores the application of reinforcement learning and multi-agent reinforcement learning (MARL) in portfolio management, emphasizing the applicability and increased portfolio performance via strategy diversification. A key contribution of this work is the development and evaluation of a MARL environment incorporating novel diversity measures, correlation, and total variation distance. The findings reveal that while fine-tuned single-agent RL models can demonstrate strong performance, roughly tuned MARL models with diverse agents reflecting a "portfolio of portfolios" paradigm show improved action diversification and portfolio performances. The work also highlights the critical role of long-term robustness testing, algorithm- and problem-specific hyperparameter optimization, and the challenges of adapting MARL methods to financial contexts. This work contributes to the RL research in portfolio management by exploring the use of MARL in portfolio management and discussing the limitations and future work directions.

Traditional path-planning methods for mobile robots typically focus on avoiding obstacles but often fall short when obstacles block the path to the goal. This paper addresses the challenge of Navigation Among Movable Obstacles (NAMO), where a single robot can reposition obstacles to create previously inaccessible pathways. We introduce SVG-MPPI, a novel approach that integrates semantics to incorporate continuous movability into both path planning and local control strategies, allowing the robot to navigate cluttered environments by moving obstacles as needed to reach its goal.

SVG-MPPI refines the traditional Visibility Graph (VG) method by introducing the Semantic Visibility Graph (SVG). In this advanced approach, additional nodes are placed near movable obstacles, with the movability of these obstacles assessed based on their estimated mass—a key semantic property defining movability. This enables the modeling of potential passages through these barriers. The cost of push actions needed to navigate to these additional nodes is incorporated into the path-finding algorithm, eliminating the need for explicit obstacle placement and reducing the overhead of custom task planning. The local control strategy employs Model Predictive Path Integral (MPPI), which uses the IsaacGym physics engine to simulate robot and obstacle state transitions. MPPI minimizes trajectory contact forces as part of its objective, thereby reducing the push actions executed by the robot. The system supports optional replanning by continuously evaluating obstacle movability and adjusting the path if actual conditions deviate from initial estimates.

Our solution was evaluated through both qualitative and quantitative experiments. Qualitative experiments demonstrated the algorithm’s success in a simulated environment, where it effectively handled path execution and replanning scenarios. In real-world testing, an omnidirectional robot successfully pushed an obstacle and established a path to a goal. Quantitative analyses compared SVG-MPPI with its unaltered planning method (VG), which has no notion of movability, and Rapidly-exploring Random Tree (RRT) adapted to include binary movability. Each of these path planning methods was equipped with MPPI, which had a similar notion of movability as the planner. Results showed that SVG-MPPI consistently outperformed both VG and RRT in terms of path planning success rate and execution success rate. Furthermore, SVG-MPPI exhibited lower cumulative contact forces over the trajectory, indicating that the integration of continuous movability allowed the algorithm to select paths of least resistance and navigate around obstacles where possible.

Overall, SVG-MPPI represents a significant advancement in NAMO planning by offering a cohesive solution that addresses the complexities of navigating random, cluttered environments with movable obstacles. By prioritizing direct progress toward the goal while repositioning obstacles, SVG-MPPI introduces a novel, integrated approach with promising results and substantial potential for future research and development.

Due to the challenges of traffic congestion, pollution, and low transportation efficiency in urban areas, Shared Automated Vehicles (SAVs) are considered a solution that can integrate into existing transportation frameworks to improve traffic performance and environmental sustainability. Currently, there is a research gap in managing the fleet size of SAVs under different matching strategies. Although the literature on the operational and environmental benefits of SAVs is growing, few studies systematically analyze how different fleet management strategies affect service levels and road congestion. This thesis fills this gap by exploring the impacts of Immediate Decision and Batch Offer strategies on fleet size and their corresponding effects on urban traffic performance.

The thesis focuses on the integration of SAV with mixed mobility systems, using SAVs to entirely replace current Mobility on Demand (MoD) services, employing an agent-based simulation method to study the impacts of fleet size and matching strategies on urban traffic performance. This research employs a case study approach, selecting the Haidian District in Beijing as the research area. It uses an innovative demand generation model based on travel distances as simulation request inputs. The model utilizes travel distance distribution functions based on historical data, road network information, and taxi GPS data to generate traffic demand based on network nodes through traffic zone division, probability density calculation, and travel distribution. The model employs the K-means clustering algorithm to divide traffic areas based on the density of traffic nodes rather than specific road network conditions, a macroscopic, low-resolution method with lower computational complexity suitable for large-scale road networks.

The mode choice model in this study analyzes how passengers make decisions between different modes of travel. The model, using a discrete choice framework, studies how passengers choose between SAVs(Shared Automated Vehicles) and POVs(privately operated vehicles). Employing the Logit model, the study considers several key factors influencing travel choice, including travel time, comfort, reliability, and environmental impact, to provide mode selection in simulations. This part emphasizes the importance of travelers' subjective perceptions.

The application of simulation and the Macroscopic Fundamental Diagram (MFD) are central methodologies of this study. In the simulation analysis part, this research uses the agent-based simulation tool FleetPy, focusing on evaluating two key matching strategies: Immediate Decision Simulation (IDS) and Batch Offer Simulation (BOS). By simulating different fleet sizes and matching strategies, it explores the dispatching and operational performance of Shared Autonomous Vehicles in urban road networks and compares them with traditional taxi systems.

The findings indicate that replacing taxi systems with an SAV system can significantly improve fleet efficiency, achieving the same service level with a smaller fleet size. In both small and large fleet scenarios, the SAV system demonstrates higher operational efficiency. In scenarios with smaller fleet sizes, the Batch Offer (BOS) strategy provides higher service levels. By batch processing and optimizing multiple requests, it can effectively improve the success rate of matching and reduce waiting times, while Immediate Decision (IDS) performs better in terms of service timeliness. Simulation results also show that as the fleet size increases, the advantages of the IDS strategy become apparent but may lead to wastage of vehicle resources.

The need for automation of unmanned aerial vehicles (UAVs) rises with the increase of inexperienced operators. Extensive research has gone into automating and understanding multirotor UAVs and fixed wing UAVs. As vertical take off and landing (VTOL) UAVs are a relatively new category, there is ample opportunity for research in automation and understanding. This report focuses on automating the landing of a specific VTOL UAV: the
Marlyn, created by ATMOS UAV.

Marlyn is a tailsitter, meaning that the centre of mass during the hover phase is relatively far away from the landing rods. This can lead to tip overs during landing. The centre of mass alone will cause a tip over moment at a pitch angle of θ ≤ −32.5 degrees. Experiments have been performed to find the relation between the incoming wind speed and the pitching moment generated by the wind. Adding the found pitching moment, the tip over limit angle decreases to θ ≤ −47.5 degrees. This corresponds to a wind speed of 8.4 m/s.

A surprising find was that not the wind nor the centre of mass was the biggest culprit for tip overs. The biggest factor was the reduction in controllability. Marlyn’s bottom tail motor is in line with the landing rod that first touches the ground. This gives Marlyn no way to counter a forward tip over once it has been set in motion.

The firmware of Marlyn (PX4) comes with a complete simulation environment based on Gazebo. This simulation runs a simple aerodynamic model. The parameters for this model are based on estimations gained from XFLR. To determine and measure the difference between the simulation and reality, a sensitivity study is done. This study shows the influence of the parameters gained from XFLR on the lift and drag forces. After the sensitivity study, the values from XFLR are compared to the experimental data used to find the relation between the incoming wind speed and the pitching moment generated by the wind. The absolute values deviated significantly, but the relative differences between the values was more or less constant.

To counter the tip overs, three strategies are proposed. The first strategy revolves around detecting the moment the bottom landing rod touches the ground. By killing the motors, the wind will always prevent the forward tip over by blowing Marlyn upright. Using the inertial measurement unit inside the autopilot to determine the impact showed to be non feasible given the current design. The impact forces were not significant enough on all surfaces to reliably be used as a trigger to kill the motors. Mechanical switches have been shown in previous research to not work reliably in all conditions, mainly failing in muddy or sandy environments. Range finders and vision based distance sensors showed the most promising, if it can be guaranteed that they always point straight down.

The second strategy is based on generating a moment using the touch down impact to fling Marlyn upright. To generate this moment, a horizontal impact is generated by allowing Marlyn to drift with the wind in a controlled manner. The path generated for this could be done using artificial potential fields. These can be used to safely find the optimal path to the ground while staying within safety bounds. To validate the feasibility of this strategy, a simple inverted pendulum simulation is used. The Gazebo based simulation could not be used as is, since the control loops that try to stabilise Marlyn would try to counter the moment enerated by the impact. The simple simulation has shown that there is merit in this strategy, but a lot of further research is required to validate that this also works in the real world.

The last strategy proposed tries to reduce the pitch angle during landing. This can be achieved via two methods that are discussed. The first rotates Marlyn so that the wings are not tangent but parallel to the wind. This method was discarded due to the actuators not being able to keep Marlyn in that unstable state for prolonged duration. The second method uses Marlyn’s ability to rotate her wing motors independently. By rotating both into the wind, these motors can generate a force to counter the drag caused by the wind. This in turn leads to a smaller pitch angle required to remain stationary. The Gazebo based simulation is used to validate this method’s feasibility. The wind speed at which the previously found tip over pitch angle was crossed was increased from 8.4 m/s to over 9 m/s. Due to instabilities in the simulation, wind speeds larger than 9 m/s could not be tested.

All strategies discussed have promising options. The most promising strategy to counter tip overs during the landing of the VTOL tailsitter UAV Marlyn is: Rotating the wing motors into the incoming wind to reduce the required pitch angle to remain stationary. Further research is required to validate this strategy in real world scenarios

This paper proposes a novel framework that combines both planning and learning-based trajectory generation methods to handle complex robotic assembly tasks. The framework utilizes MoveIt! for planning large-scale reaching motions and Dynamic Movement Primitives (DMPs) for precise grasping and placing movements, with both methods integrated into a single system controlled by a behavior tree. An impedance controller is employed to ensure smooth and safe execution of the generated trajectories, particularly in scenarios that involve human interaction.

The proposed framework was evaluated within the context of the European Space Agency-funded Rhizome project, which focuses on off-earth habitat construction. The project involves assembling habitats using custom-designed Voronoi-shaped building blocks, which were also utilized in experiments to test the framework. The results showed that combining planning for large-reaching motions with DMPs for detailed movements effectively addressed the limitations of each individual method, delivering a flexible and robust solution to the challenges of robotic assembly.

Modern control systems require control strategies that can handle multiple levels of abstraction while providing formal guarantees of system behavior. Traditional hierarchical control approaches often lack formal interfaces between different layers of abstraction, making it challenging to ensure system-wide properties and extend implementations across different platforms.

This thesis presents a contract-based hierarchical control framework that provides formal guarantees across different layers of abstraction. Building upon recent developments in vertical contracts with heterogeneous refinements, we develop a three-layer control architecture for robot surveillance tasks, decomposing the problem into motion planning, trajectory generation, and tracking control. Each layer operates at its own time scale and uses appropriate model abstractions, with contracts ensuring system-wide guarantees while vertical system relationships enable proper interfacing between layers.

We validate our framework through both simulation studies and physical implementation on Elisa3 robots. The simulation results demonstrate the framework's ability to satisfy high-level temporal logic specifications while maintaining guarantees at each layer. The physical implementation reveals practical challenges in transitioning from simulation to hardware, leading to adaptations in the control architecture that maintain theoretical guarantees while accommodating hardware constraints.

Reinforcement learning, especially deep reinforcement learning, has made many advances in the last decade. Similarly, great strides have been made in multi-agent reinforcement learning. Systems of cooperative autonomous robots are increasingly being used, for which multi-agent reinforcement learning can be used as a training method. However, the curse of dimensionality remains a problem for the computational speed of the learning algorithm and the bandwidth of communication channels. This research will focus mainly on reducing the problem of overloading communication channels by trying to reduce the number of communications. This is possible since it is usually unnecessary for every agent to communicate with every other agent constantly.

To do this, we use some ideas by Daniel Jarne Ornia. The first is to reduce communications in a multi-agent reinforcement learning system by treating it as an event-triggered control problem. This method uses so-called robustness surrogates as an equivalent to a Lyapunov function to determine if a communication can be skipped without decreasing the performance more than some tolerance. The second is a method to increase the observational robustness of a policy by using lexicographic reinforcement learning.

We aim to combine these ideas and trade the additional observational robustness for decreased communications. We also want to test whether additional observational robustness can help mitigate the sim-to-real gap. We implement this method for the multi-agent deep deterministic policy gradient algorithm and perform tests on a variant of the predator-prey domain in increasingly more realistic simulations.

We found that the combination of this robust policy and the robustness surrogates method does enable the agents to achieve the same return while communicating less. Unfortunately, our research shows that the observational robustness obtained using lexicographic reinforcement learning, does not help mitigate the sim-to-real gap.

Automated vehicles represent an exciting advancement in transportation, offering a range of benefits that have the potential to revolutionize how we travel. They can improve safety, efficiency, accessibility, and sustainability, holding promise for transforming our cities and communities. However, generating safe, comfortable, and efficient motion plans, especially in interactive scenarios like forced lane merging, remains a significant challenge in the field.

Lane merging is a pivotal skill for automated vehicles, as it frequently involves changing lanes to reach a destination. For instance, when approaching an intersection, a vehicle might need to merge into a specific lane beforehand to execute a turn later. Traditional pipelines in automated driving typically decouple prediction and planning. They assume perfect upstream prediction and generate robust motion plans to avoid collisions with multi-modal predictions. However, in dense traffic conditions, conservative planning might hinder the ego vehicle from merging effectively, resulting in it becoming stuck. This underscores the necessity of combining prediction and planning, a concept we term interaction-aware planning algorithms.

The first major contribution of this thesis is an efficient game-theoretic behavior planner that captures interactions under different behavior modes. In this approach, we represent the behaviors of the vehicles as actions in a matrix game and select the Nash equilibrium to capture their mutual influence. To generate the cost of the action pairs, we model the merging process as a gap selection process and evaluate the trajectories generated by interactive models. The effectiveness of the proposed planner is validated in the high-fidelity CARLA simulator.

In the real world, human drivers may not always adhere rationally to the equilibrium of a game model. They could choose a behavior mode different from the game theory solution. Therefore, it might be more beneficial to consider different motion modes simultaneously, rather than favoring the "most likely" one while neglecting the others.

For this purpose, we also explore the usage of Branch Model Predictive Control (B-MPC) in this thesis. By predicting the motion of the surrounding vehicle as a scenario tree, the B-MPC approach can generate a trajectory tree as a motion plan. By executing only the root node, the ego vehicle can consider different future scenarios simultaneously and plan contingency motions. We further extend the B-MPC approach by incorporating interactive policies, using different solving schemes, and including collision avoidance constraints that consider the orientations of the vehicles. The effectiveness of these proposed methods was validated through experiments conducted in a handcrafted lightweight simulator.

Overall, this thesis focuses on developing interaction-aware planning methods to facilitate safe, successful, and comfortable lane merging scenarios. Nevertheless, the major limitations lie in the modeling error and the potential long-tail issues. To address these challenges and further improve motion planning, future work could explore data-driven (learning-based) approaches that leverage real-world driver behavior to generate more informed and adaptable motion plans.

As Autonomous Vehicles (AVs) navigate through dynamic and constantly changing environments, it is crucial that they take into account the impact of their actions on the decisions of others for safe and efficient interaction with humans. In doing so, they need to anticipate how humans will behave in different situations based on their intentions. This work aims to address these challenges by proposing an expressive game-theoretic framework for modeling the interactions between AVs and human drivers as a multi-agent dynamic game, in which each agent seeks to optimize their respective objective. The optimization problem is solved by obtaining the Nash equilibrium, which accounts for the potential non-cooperative behavior of human drivers. To incorporate the notion of intention into the game-theoretic formulation, we introduce for each agent a parameter known as Social Value Orientation (SVO), reflecting the degree to which an agent is willing to prioritize the welfare of others over its own. We then develop efficient methods to solve this nonlinear optimization problem in a receding-horizon fashion given the agents' SVOs. However, cluttered traffic scenarios are typically characterized by uncertainty regarding the intentions of other traffic participants due to noisy sensor data, and multiple equally admissible equilibrium strategies that humans may adapt to achieve their objective. Therefore, an approximate Bayesian inference method is developed to infer the intentions of the surrounding human participants by estimating the likelihood of SVO based on newly received state observations. We then integrate the estimation module into the game-theoretic planning module in a combined framework and evaluate its predictive performance against algorithms that ignore these sources of uncertainty in two simulated traffic scenarios; ramp-merging at a highway and crossing at uncontrolled intersections. Our results show that the proposed inference method exhibits superior performance compared to all other approaches, with the average prediction error approaching zero. This implies that dynamically changing the SVO values, while planning, effectively captures the true intentions of the surrounding agents.

Creating autonomous Micro Aerial Vehicles for executing complex missions poses various challenges, including safe navigation in the presence of external wind disturbances. Most current navigation methods handle external wind disturbances through real-time estimation and rejection algorithms in the control stage, but lack safety guarantees in strong winds. Recent robust methods provide safety guarantees but can be overly conservative. With the availability of more powerful computing devices, data-driven control algorithms are becoming increasingly feasible. Combining Gaussian Process models with Model Predictive Control has shown to enhance safety and performance in various control applications. Moreover, Model Predictive Control has been extended to solve more complex optimization algorithms that combine trajectory generation and tracking, preventing reference trajectories that are risky and challenging to control in the presence of disturbances.

This research aims to improve quadrotor navigation in windy environments by using Gaussian Processes to model wind disturbances. The Gaussian Process model is integrated with a Model Predictive Contouring Control formulation that combines trajectory generation and control into one optimization problem. A nominal model of the quadrotor is derived and the Gaussian Process disturbance model is trained with the quadrotor position as input and wind disturbance as output. The wind disturbance map, along with the nominal model, is implemented in the Model Predictive Contouring Controller to consider both the mean and uncertainty of the resulting probabilistic model.

This study validates the use of Gaussian Processes to model complex wind disturbances in quadrotor navigation. The wind disturbance map is trained from available state information, with data collected using an optimal exploration design to minimize uncertainty and reduce exploration times. The hyperparameters involved in training the Gaussian Process model are discussed and the implementation of sparse Gaussian Processes is outlined to make it real-time feasible for the Model Predictive Contouring Control formulation. Including the prediction model by incorporating chance constraints results in improved tracking and increased robustness in cluttered environments compared to the nominal Model Predictive Contouring Control formulation. The proposed algorithm is shown to outperform state-of-the-art methods for safe quadrotor navigation in windy and cluttered environments, being able to handle more complex wind fields than existing methods while also being less conservative.

Demand for online grocer Picnic has increased exponentially over the past years, and their truck transport operation must scale with it. Given the resource constraints at all warehouses, as well as other specific restrictions, this poses a Multi Depot Pickup and Delivery Problem with Resource Constraints, for which no good solutions are found to date. The goal of this thesis is to develop an algorithm that provides good solutions to this problem within 30 minutes.

This thesis presents three solution methods to solve this problem. The first is an Adaptive Large Neighbourhood Search (ALNS) algorithm closely related to earlier research. We propose a mechanism with only linear time additional complexity to impose the resource constraint. We also demonstrate ways to take additional constraints into account, such as minimum route duration and driver switches. For the second method, dubbed ALNS+LS, we extend the ALNS with local search heuristics to enhance its performance. As a third method, we propose a matheuristic novel to this specific problem class. This consists of the ALNS+LS algorithm applied to the problem without resource constraints in the first phase, and imposing the constraints using a Constraint Programming model in the second phase. We show that the ALNS+LS outperforms the other two algorithms on a real-life-inspired benchmark set, and that the matheuristic comes close to the ALNS+LS for small instances. We finally show that the full problem is best solved by decomposing it into four parts and solving these separately.