Reinforcement learning (RL) is a powerful tool where the agents – or “robots” can learn from the environment based on their actions. Reinforcement learning approaches were found successful in combining predicting stock returns and portfolio allocation. Diversification is a critical element for achieving high portfolio returns with a lower level of risk. This work explores the application of reinforcement learning and multi-agent reinforcement learning (MARL) in portfolio management, emphasizing the applicability and increased portfolio performance via strategy diversification. A key contribution of this work is the development and evaluation of a MARL environment incorporating novel diversity measures, correlation, and total variation distance. The findings reveal that while fine-tuned single-agent RL models can demonstrate strong performance, roughly tuned MARL models with diverse agents reflecting a "portfolio of portfolios" paradigm show improved action diversification and portfolio performances. The work also highlights the critical role of long-term robustness testing, algorithm- and problem-specific hyperparameter optimization, and the challenges of adapting MARL methods to financial contexts. This work contributes to the RL research in portfolio management by exploring the use of MARL in portfolio management and discussing the limitations and future work directions.

Traditional path-planning methods for mobile robots typically focus on avoiding obstacles but often fall short when obstacles block the path to the goal. This paper addresses the challenge of Navigation Among Movable Obstacles (NAMO), where a single robot can reposition obstacles to create previously inaccessible pathways. We introduce SVG-MPPI, a novel approach that integrates semantics to incorporate continuous movability into both path planning and local control strategies, allowing the robot to navigate cluttered environments by moving obstacles as needed to reach its goal.

SVG-MPPI refines the traditional Visibility Graph (VG) method by introducing the Semantic Visibility Graph (SVG). In this advanced approach, additional nodes are placed near movable obstacles, with the movability of these obstacles assessed based on their estimated mass—a key semantic property defining movability. This enables the modeling of potential passages through these barriers. The cost of push actions needed to navigate to these additional nodes is incorporated into the path-finding algorithm, eliminating the need for explicit obstacle placement and reducing the overhead of custom task planning. The local control strategy employs Model Predictive Path Integral (MPPI), which uses the IsaacGym physics engine to simulate robot and obstacle state transitions. MPPI minimizes trajectory contact forces as part of its objective, thereby reducing the push actions executed by the robot. The system supports optional replanning by continuously evaluating obstacle movability and adjusting the path if actual conditions deviate from initial estimates.

Our solution was evaluated through both qualitative and quantitative experiments. Qualitative experiments demonstrated the algorithm’s success in a simulated environment, where it effectively handled path execution and replanning scenarios. In real-world testing, an omnidirectional robot successfully pushed an obstacle and established a path to a goal. Quantitative analyses compared SVG-MPPI with its unaltered planning method (VG), which has no notion of movability, and Rapidly-exploring Random Tree (RRT) adapted to include binary movability. Each of these path planning methods was equipped with MPPI, which had a similar notion of movability as the planner. Results showed that SVG-MPPI consistently outperformed both VG and RRT in terms of path planning success rate and execution success rate. Furthermore, SVG-MPPI exhibited lower cumulative contact forces over the trajectory, indicating that the integration of continuous movability allowed the algorithm to select paths of least resistance and navigate around obstacles where possible.

Overall, SVG-MPPI represents a significant advancement in NAMO planning by offering a cohesive solution that addresses the complexities of navigating random, cluttered environments with movable obstacles. By prioritizing direct progress toward the goal while repositioning obstacles, SVG-MPPI introduces a novel, integrated approach with promising results and substantial potential for future research and development.

Due to the challenges of traffic congestion, pollution, and low transportation efficiency in urban areas, Shared Automated Vehicles (SAVs) are considered a solution that can integrate into existing transportation frameworks to improve traffic performance and environmental sustainability. Currently, there is a research gap in managing the fleet size of SAVs under different matching strategies. Although the literature on the operational and environmental benefits of SAVs is growing, few studies systematically analyze how different fleet management strategies affect service levels and road congestion. This thesis fills this gap by exploring the impacts of Immediate Decision and Batch Offer strategies on fleet size and their corresponding effects on urban traffic performance.

The thesis focuses on the integration of SAV with mixed mobility systems, using SAVs to entirely replace current Mobility on Demand (MoD) services, employing an agent-based simulation method to study the impacts of fleet size and matching strategies on urban traffic performance. This research employs a case study approach, selecting the Haidian District in Beijing as the research area. It uses an innovative demand generation model based on travel distances as simulation request inputs. The model utilizes travel distance distribution functions based on historical data, road network information, and taxi GPS data to generate traffic demand based on network nodes through traffic zone division, probability density calculation, and travel distribution. The model employs the K-means clustering algorithm to divide traffic areas based on the density of traffic nodes rather than specific road network conditions, a macroscopic, low-resolution method with lower computational complexity suitable for large-scale road networks.

The mode choice model in this study analyzes how passengers make decisions between different modes of travel. The model, using a discrete choice framework, studies how passengers choose between SAVs(Shared Automated Vehicles) and POVs(privately operated vehicles). Employing the Logit model, the study considers several key factors influencing travel choice, including travel time, comfort, reliability, and environmental impact, to provide mode selection in simulations. This part emphasizes the importance of travelers' subjective perceptions.

The application of simulation and the Macroscopic Fundamental Diagram (MFD) are central methodologies of this study. In the simulation analysis part, this research uses the agent-based simulation tool FleetPy, focusing on evaluating two key matching strategies: Immediate Decision Simulation (IDS) and Batch Offer Simulation (BOS). By simulating different fleet sizes and matching strategies, it explores the dispatching and operational performance of Shared Autonomous Vehicles in urban road networks and compares them with traditional taxi systems.

The findings indicate that replacing taxi systems with an SAV system can significantly improve fleet efficiency, achieving the same service level with a smaller fleet size. In both small and large fleet scenarios, the SAV system demonstrates higher operational efficiency. In scenarios with smaller fleet sizes, the Batch Offer (BOS) strategy provides higher service levels. By batch processing and optimizing multiple requests, it can effectively improve the success rate of matching and reduce waiting times, while Immediate Decision (IDS) performs better in terms of service timeliness. Simulation results also show that as the fleet size increases, the advantages of the IDS strategy become apparent but may lead to wastage of vehicle resources.

The need for automation of unmanned aerial vehicles (UAVs) rises with the increase of inexperienced operators. Extensive research has gone into automating and understanding multirotor UAVs and fixed wing UAVs. As vertical take off and landing (VTOL) UAVs are a relatively new category, there is ample opportunity for research in automation and understanding. This report focuses on automating the landing of a specific VTOL UAV: the
Marlyn, created by ATMOS UAV.

Marlyn is a tailsitter, meaning that the centre of mass during the hover phase is relatively far away from the landing rods. This can lead to tip overs during landing. The centre of mass alone will cause a tip over moment at a pitch angle of θ ≤ −32.5 degrees. Experiments have been performed to find the relation between the incoming wind speed and the pitching moment generated by the wind. Adding the found pitching moment, the tip over limit angle decreases to θ ≤ −47.5 degrees. This corresponds to a wind speed of 8.4 m/s.

A surprising find was that not the wind nor the centre of mass was the biggest culprit for tip overs. The biggest factor was the reduction in controllability. Marlyn’s bottom tail motor is in line with the landing rod that first touches the ground. This gives Marlyn no way to counter a forward tip over once it has been set in motion.

The firmware of Marlyn (PX4) comes with a complete simulation environment based on Gazebo. This simulation runs a simple aerodynamic model. The parameters for this model are based on estimations gained from XFLR. To determine and measure the difference between the simulation and reality, a sensitivity study is done. This study shows the influence of the parameters gained from XFLR on the lift and drag forces. After the sensitivity study, the values from XFLR are compared to the experimental data used to find the relation between the incoming wind speed and the pitching moment generated by the wind. The absolute values deviated significantly, but the relative differences between the values was more or less constant.

To counter the tip overs, three strategies are proposed. The first strategy revolves around detecting the moment the bottom landing rod touches the ground. By killing the motors, the wind will always prevent the forward tip over by blowing Marlyn upright. Using the inertial measurement unit inside the autopilot to determine the impact showed to be non feasible given the current design. The impact forces were not significant enough on all surfaces to reliably be used as a trigger to kill the motors. Mechanical switches have been shown in previous research to not work reliably in all conditions, mainly failing in muddy or sandy environments. Range finders and vision based distance sensors showed the most promising, if it can be guaranteed that they always point straight down.

The second strategy is based on generating a moment using the touch down impact to fling Marlyn upright. To generate this moment, a horizontal impact is generated by allowing Marlyn to drift with the wind in a controlled manner. The path generated for this could be done using artificial potential fields. These can be used to safely find the optimal path to the ground while staying within safety bounds. To validate the feasibility of this strategy, a simple inverted pendulum simulation is used. The Gazebo based simulation could not be used as is, since the control loops that try to stabilise Marlyn would try to counter the moment enerated by the impact. The simple simulation has shown that there is merit in this strategy, but a lot of further research is required to validate that this also works in the real world.

The last strategy proposed tries to reduce the pitch angle during landing. This can be achieved via two methods that are discussed. The first rotates Marlyn so that the wings are not tangent but parallel to the wind. This method was discarded due to the actuators not being able to keep Marlyn in that unstable state for prolonged duration. The second method uses Marlyn’s ability to rotate her wing motors independently. By rotating both into the wind, these motors can generate a force to counter the drag caused by the wind. This in turn leads to a smaller pitch angle required to remain stationary. The Gazebo based simulation is used to validate this method’s feasibility. The wind speed at which the previously found tip over pitch angle was crossed was increased from 8.4 m/s to over 9 m/s. Due to instabilities in the simulation, wind speeds larger than 9 m/s could not be tested.

All strategies discussed have promising options. The most promising strategy to counter tip overs during the landing of the VTOL tailsitter UAV Marlyn is: Rotating the wing motors into the incoming wind to reduce the required pitch angle to remain stationary. Further research is required to validate this strategy in real world scenarios

This paper proposes a novel framework that combines both planning and learning-based trajectory generation methods to handle complex robotic assembly tasks. The framework utilizes MoveIt! for planning large-scale reaching motions and Dynamic Movement Primitives (DMPs) for precise grasping and placing movements, with both methods integrated into a single system controlled by a behavior tree. An impedance controller is employed to ensure smooth and safe execution of the generated trajectories, particularly in scenarios that involve human interaction.

The proposed framework was evaluated within the context of the European Space Agency-funded Rhizome project, which focuses on off-earth habitat construction. The project involves assembling habitats using custom-designed Voronoi-shaped building blocks, which were also utilized in experiments to test the framework. The results showed that combining planning for large-reaching motions with DMPs for detailed movements effectively addressed the limitations of each individual method, delivering a flexible and robust solution to the challenges of robotic assembly.

Modern control systems require control strategies that can handle multiple levels of abstraction while providing formal guarantees of system behavior. Traditional hierarchical control approaches often lack formal interfaces between different layers of abstraction, making it challenging to ensure system-wide properties and extend implementations across different platforms.

This thesis presents a contract-based hierarchical control framework that provides formal guarantees across different layers of abstraction. Building upon recent developments in vertical contracts with heterogeneous refinements, we develop a three-layer control architecture for robot surveillance tasks, decomposing the problem into motion planning, trajectory generation, and tracking control. Each layer operates at its own time scale and uses appropriate model abstractions, with contracts ensuring system-wide guarantees while vertical system relationships enable proper interfacing between layers.

We validate our framework through both simulation studies and physical implementation on Elisa3 robots. The simulation results demonstrate the framework's ability to satisfy high-level temporal logic specifications while maintaining guarantees at each layer. The physical implementation reveals practical challenges in transitioning from simulation to hardware, leading to adaptations in the control architecture that maintain theoretical guarantees while accommodating hardware constraints.

Automated vehicles represent an exciting advancement in transportation, offering a range of benefits that have the potential to revolutionize how we travel. They can improve safety, efficiency, accessibility, and sustainability, holding promise for transforming our cities and communities. However, generating safe, comfortable, and efficient motion plans, especially in interactive scenarios like forced lane merging, remains a significant challenge in the field.

Lane merging is a pivotal skill for automated vehicles, as it frequently involves changing lanes to reach a destination. For instance, when approaching an intersection, a vehicle might need to merge into a specific lane beforehand to execute a turn later. Traditional pipelines in automated driving typically decouple prediction and planning. They assume perfect upstream prediction and generate robust motion plans to avoid collisions with multi-modal predictions. However, in dense traffic conditions, conservative planning might hinder the ego vehicle from merging effectively, resulting in it becoming stuck. This underscores the necessity of combining prediction and planning, a concept we term interaction-aware planning algorithms.

The first major contribution of this thesis is an efficient game-theoretic behavior planner that captures interactions under different behavior modes. In this approach, we represent the behaviors of the vehicles as actions in a matrix game and select the Nash equilibrium to capture their mutual influence. To generate the cost of the action pairs, we model the merging process as a gap selection process and evaluate the trajectories generated by interactive models. The effectiveness of the proposed planner is validated in the high-fidelity CARLA simulator.

In the real world, human drivers may not always adhere rationally to the equilibrium of a game model. They could choose a behavior mode different from the game theory solution. Therefore, it might be more beneficial to consider different motion modes simultaneously, rather than favoring the "most likely" one while neglecting the others.

For this purpose, we also explore the usage of Branch Model Predictive Control (B-MPC) in this thesis. By predicting the motion of the surrounding vehicle as a scenario tree, the B-MPC approach can generate a trajectory tree as a motion plan. By executing only the root node, the ego vehicle can consider different future scenarios simultaneously and plan contingency motions. We further extend the B-MPC approach by incorporating interactive policies, using different solving schemes, and including collision avoidance constraints that consider the orientations of the vehicles. The effectiveness of these proposed methods was validated through experiments conducted in a handcrafted lightweight simulator.

Overall, this thesis focuses on developing interaction-aware planning methods to facilitate safe, successful, and comfortable lane merging scenarios. Nevertheless, the major limitations lie in the modeling error and the potential long-tail issues. To address these challenges and further improve motion planning, future work could explore data-driven (learning-based) approaches that leverage real-world driver behavior to generate more informed and adaptable motion plans.

Reinforcement learning, especially deep reinforcement learning, has made many advances in the last decade. Similarly, great strides have been made in multi-agent reinforcement learning. Systems of cooperative autonomous robots are increasingly being used, for which multi-agent reinforcement learning can be used as a training method. However, the curse of dimensionality remains a problem for the computational speed of the learning algorithm and the bandwidth of communication channels. This research will focus mainly on reducing the problem of overloading communication channels by trying to reduce the number of communications. This is possible since it is usually unnecessary for every agent to communicate with every other agent constantly.

To do this, we use some ideas by Daniel Jarne Ornia. The first is to reduce communications in a multi-agent reinforcement learning system by treating it as an event-triggered control problem. This method uses so-called robustness surrogates as an equivalent to a Lyapunov function to determine if a communication can be skipped without decreasing the performance more than some tolerance. The second is a method to increase the observational robustness of a policy by using lexicographic reinforcement learning.

We aim to combine these ideas and trade the additional observational robustness for decreased communications. We also want to test whether additional observational robustness can help mitigate the sim-to-real gap. We implement this method for the multi-agent deep deterministic policy gradient algorithm and perform tests on a variant of the predator-prey domain in increasingly more realistic simulations.

We found that the combination of this robust policy and the robustness surrogates method does enable the agents to achieve the same return while communicating less. Unfortunately, our research shows that the observational robustness obtained using lexicographic reinforcement learning, does not help mitigate the sim-to-real gap.

As Autonomous Vehicles (AVs) navigate through dynamic and constantly changing environments, it is crucial that they take into account the impact of their actions on the decisions of others for safe and efficient interaction with humans. In doing so, they need to anticipate how humans will behave in different situations based on their intentions. This work aims to address these challenges by proposing an expressive game-theoretic framework for modeling the interactions between AVs and human drivers as a multi-agent dynamic game, in which each agent seeks to optimize their respective objective. The optimization problem is solved by obtaining the Nash equilibrium, which accounts for the potential non-cooperative behavior of human drivers. To incorporate the notion of intention into the game-theoretic formulation, we introduce for each agent a parameter known as Social Value Orientation (SVO), reflecting the degree to which an agent is willing to prioritize the welfare of others over its own. We then develop efficient methods to solve this nonlinear optimization problem in a receding-horizon fashion given the agents' SVOs. However, cluttered traffic scenarios are typically characterized by uncertainty regarding the intentions of other traffic participants due to noisy sensor data, and multiple equally admissible equilibrium strategies that humans may adapt to achieve their objective. Therefore, an approximate Bayesian inference method is developed to infer the intentions of the surrounding human participants by estimating the likelihood of SVO based on newly received state observations. We then integrate the estimation module into the game-theoretic planning module in a combined framework and evaluate its predictive performance against algorithms that ignore these sources of uncertainty in two simulated traffic scenarios; ramp-merging at a highway and crossing at uncontrolled intersections. Our results show that the proposed inference method exhibits superior performance compared to all other approaches, with the average prediction error approaching zero. This implies that dynamically changing the SVO values, while planning, effectively captures the true intentions of the surrounding agents.

Creating autonomous Micro Aerial Vehicles for executing complex missions poses various challenges, including safe navigation in the presence of external wind disturbances. Most current navigation methods handle external wind disturbances through real-time estimation and rejection algorithms in the control stage, but lack safety guarantees in strong winds. Recent robust methods provide safety guarantees but can be overly conservative. With the availability of more powerful computing devices, data-driven control algorithms are becoming increasingly feasible. Combining Gaussian Process models with Model Predictive Control has shown to enhance safety and performance in various control applications. Moreover, Model Predictive Control has been extended to solve more complex optimization algorithms that combine trajectory generation and tracking, preventing reference trajectories that are risky and challenging to control in the presence of disturbances.

This research aims to improve quadrotor navigation in windy environments by using Gaussian Processes to model wind disturbances. The Gaussian Process model is integrated with a Model Predictive Contouring Control formulation that combines trajectory generation and control into one optimization problem. A nominal model of the quadrotor is derived and the Gaussian Process disturbance model is trained with the quadrotor position as input and wind disturbance as output. The wind disturbance map, along with the nominal model, is implemented in the Model Predictive Contouring Controller to consider both the mean and uncertainty of the resulting probabilistic model.

This study validates the use of Gaussian Processes to model complex wind disturbances in quadrotor navigation. The wind disturbance map is trained from available state information, with data collected using an optimal exploration design to minimize uncertainty and reduce exploration times. The hyperparameters involved in training the Gaussian Process model are discussed and the implementation of sparse Gaussian Processes is outlined to make it real-time feasible for the Model Predictive Contouring Control formulation. Including the prediction model by incorporating chance constraints results in improved tracking and increased robustness in cluttered environments compared to the nominal Model Predictive Contouring Control formulation. The proposed algorithm is shown to outperform state-of-the-art methods for safe quadrotor navigation in windy and cluttered environments, being able to handle more complex wind fields than existing methods while also being less conservative.

Demand for online grocer Picnic has increased exponentially over the past years, and their truck transport operation must scale with it. Given the resource constraints at all warehouses, as well as other specific restrictions, this poses a Multi Depot Pickup and Delivery Problem with Resource Constraints, for which no good solutions are found to date. The goal of this thesis is to develop an algorithm that provides good solutions to this problem within 30 minutes.

This thesis presents three solution methods to solve this problem. The first is an Adaptive Large Neighbourhood Search (ALNS) algorithm closely related to earlier research. We propose a mechanism with only linear time additional complexity to impose the resource constraint. We also demonstrate ways to take additional constraints into account, such as minimum route duration and driver switches. For the second method, dubbed ALNS+LS, we extend the ALNS with local search heuristics to enhance its performance. As a third method, we propose a matheuristic novel to this specific problem class. This consists of the ALNS+LS algorithm applied to the problem without resource constraints in the first phase, and imposing the constraints using a Constraint Programming model in the second phase. We show that the ALNS+LS outperforms the other two algorithms on a real-life-inspired benchmark set, and that the matheuristic comes close to the ALNS+LS for small instances. We finally show that the full problem is best solved by decomposing it into four parts and solving these separately.

To safely deploy legged robots in the real world, it is necessary to provide them with the ability to reliably detect unexpected contacts and accurately estimate the corresponding contact force. Therefore, a collision detection, isolation, and identification pipeline is proposed for a quadrupedal manipulator. An approach based on band-pass filtered forces is presented which accurately detects a collision, and estimates the collision time span. Next, the colliding body link is isolated. Finally, a collision identification method accurately identifies the magnitude and direction of the force. It is robust against model inaccuracies, unmodeled loads and any other potential source of disturbances acting on the robot. The framework is validated using extensive hardware experiments in various scenarios summing up to 416 collisions, including trotting and additional unmodeled load on the robot.

Ongoing research in autonomous driving currently focuses on creating new applications for autonomous vehicles (AV) and connected autonomous vehicles (CAV). Specifically, motion planning and control solutions are being developed based on the combination of Artificial Potential Functions (APF) with economic Model Predictive Control (eMPC). These two methods are integrated into a new Comprehensive Predictive Control (CPC) strategy. Although preliminary research shows promising results, a performance analysis of this approach, both for AV and CAV, has not yet been published. Therefore this thesis studies the capabilities of this novel APF-eMPC framework by carrying out numerical simulations. Multiple manoeuvres and varying amounts of white noise are utilized to test the controller's limitations. For the AV part, multiple basic driving manoeuvres are simulated: lane-keeping, car-following and lane-changing. The results show that an AV based on this framework can execute these different manoeuvres without precise measurements. The CAV concept is simulated using a platoon scenario. The gap-closing behaviour of the multiple CAVs in a platoon is examined. The state-of-the-art gap-closing APF is compared with an APF based on inter-molecular dynamics and fitted on actual traffic. Various experiments are carried out using a constant time-headway in combination with different time gaps between the vehicles. The results show that the resulting behaviour by the inter-molecular APF better matches human driving behaviour and results in less dangerous gap-closing behaviour than the quadratic platoon APF. The latter has a more considerable change of lateral instability occurring. Therefore the APF based on inter-molecular dynamics and fitted on actual traffic data outperforms the APF based on a quadratic function. Lastly, it was found that the coupling between the longitudinal and lateral dynamics, often neglected in literature, cannot be ignored during platoon stability analysis.

Realistic vehicle routing problems have been highly relevant for years in a wide variety of domains. One such domain is food delivery, where well-crafted routes can reduce costs and contribute to customer satisfaction. This thesis formulates a problem variant for the restaurant meal delivery problem in order to examine the reoptimization of meal delivery routes. A novel solution algorithm that combines rule-based reinforcement learning and adaptive large neighbourhood search is used to tackle the problem. This hybrid algorithm manages to incorporate both learning and handcrafted search heuristics, as well as both offline and online computation. Analysis of the algorithmic components shows that the proposed algorithmic approach is computationally feasible for small scale reoptimization problems. Analysis also demonstrates that the reinforcement learning component currently does not improve performance with respect to the established objective function.

Research and development of motion control in the field of autonomous driving is significantly increasing nowadays. Model predictive control (MPC) is one of the most powerful and practical tools currently available. It is important to select the parameters of the MPC, such as weights, in a way so that different control objectives can be met within the desired performance constraints. The tuning procedure of the MPC variables can be achieved automatically by employing appropriate approaches. To make the tuning process more robust to different scenarios, one approach is to choose a decision-making architecture that provides guidance. This thesis therefore aims at developing a system integrating an automatic tuning method with a decision-making module. In order to achieve the objectives, Genetic Algorithm and Behavior Tree are employed on top of an existing motion planner. The motion planner is based on model predictive contouring control (MPCC) and the proposed method is tested in CARLA simulation environment. This report also highlights the limitations of the proposed automatic tuning method and gives concrete recommendations on how to deal with the shortcomings.

This thesis was aimed for the Master of Science degree in Mechanical Engineering at The Delft University of Technology. The goal of this research was to formulate an assignment algorithm for autonomous guidance, control, and navigation system for spacecraft formation flying using a Model Predictive Control approach for estimation of fuel expenditure. This report is split into three parts. First, the earth's gravitation field and its variation are discussed. The relative motion in the orbit around the earth is briefly explained. The derivation of linearized relative dynamics by Clohessy-Wiltshire is presented. In the second part, the assignment problem is explained. The majority of the literature has been using the distance between the deployed state to the target state. A new approach is discussed using fuel as the assignment parameter including the framework for the assignment parameters is established. For the third part, fuel expenditure estimation using two types of control schemes is explained. The Artificial Potential Function approach is concisely explained. The choice of Model Predictive Control over Artificial Potential Function is discussed. The aim for the third part was the formulation of the control problem using the Model Predictive Control with a linearization dynamics model of the satellites. The system model used in the MPC formulation was constructed based on the Clohessy–Wiltshire relative equations of motion. The demonstrations were established using randomized positions. The simulations were performed and estimation of the fuel expenditure was explained based on the simulation results. The proposed assignment algorithm was tested using the fuel cost estimations from the MPC-based simulations. The results showed that the algorithm performed as designed. Lastly, the possible future research options are listed out.

As autonomous driving is a popular and ever growing field of research, real world experiments provide a required manner of testing. In this thesis a driving research platform is developed, with a focus on platooning using visual messaging. These visual messages are conveyed using LED matrices. This thesis proposes two methods of LED matrix detection using YOLOv2, one using a sliding window, and one using the entire image. Furthermore two ways of distance estimation are proposed, one using the centers of the estimation bounding boxes and one using the used camera proprietary toolbox depth map. Results from an online experiment show best results from the depth map based depth estimation. The LED matrix detection using a sliding window gave generally dependable results in different environments, at the cost of being computationally demanding. The detection using the entire image provided less consistent results, but was significantly less computationally demanding. In a second offline experiment using a preannotated validation dataset as groundtruth all LED matrices were detected for all detectors. The SqueezeNet based YOLOv2 detector using a sliding window had the best results between tested detectors, with the highest intersection over union between detection and groundtruth.

Modern retail stores are increasingly implementing automated systems with the aim of assisting both the customer and employee. Much research is conducted on robotic applications within such environments; one of these applications concerns deploying an autonomous mobile manipulator to assist humans in retail stores. Such a robot requires the ability to cope with unexpected environmental changes, like encountering obstructions in retail store aisles. Generally, the first step in the pipeline of an autonomous mobile robot is focused on gathering and processing environmental information using perception sensors like cameras and LiDARs. Subsequently, this information is used to plan the mobile robot's path, which is typically done without allowing interaction with the environment. For object detection techniques, objects are first localized, after which they are classified towards their object type category. Another approach is to classify objects towards their functional categories, which are commonly referred to as affordances. Research in the field of affordance classification mostly focuses on kitchen, garden and working tools due to the availability of affordance datasets for these objects. However, no work on affordances in retail store environments has been conducted to this date. More specifically, there is no dataset publicly available that allows mobile manipulators to identify how to interact with retail store related objects in such environments. This work has investigated the adaptation of an instance segmentation network to localize and classify objects on floors of retail stores into affordance classes. These affordance classes relate to the functional capabilities of mobile manipulators, like graspable or pushable. To achieve this, an affordance dataset consisting of retail store related objects is essential. To overcome the scarcity in this data, a novel dataset consisting of 3237 images with pixel-level affordance annotations was successfully created using an automated data generation approach. This work has shown that such an approach can be used to minimize human labour in terms of acquiring annotated data drastically. A state-of-the-art instance segmentation network was trained using this synthetically generated data and was tested on both synthetic and real image data. The evaluation revealed that the use of synthetic data for training allows for inference on real image data, yet this may compromise the localization and especially the classification performance. Further, the observation of objects that are assigned to both affordance classes has introduced the aspect of the subjectivity of affordances.

With the performance of current motion planning methods being highly dependent on the quality of the perception system, robust 3D multi-object detection and tracking are vital for autonomous driving applications. Despite all the advancements in 2D and 3D object detectors, robust tracking of pedestrians in dense scenarios is still a challenging subject for small Automated Guided Vehicles (AGVs). Most research in the field of object detection and tracking focuses on autonomous cars, neglecting the design challenges that come with small AGVs.

This thesis presents a real-time multi-modal multi-pedestrian detection and tracking pipeline for small mobile robots. The framework integrates five RGB-D cameras and a LiDAR sensor to achieve real-time pedestrian detection and tracking. The system relies on state-of-the-art 2D and 3D object detectors, a sensor fusion and filtering scheme, and a 3D object tracker. Moreover, to improve detection and tracking performance, we have collected a pedestrian dataset tailored for small AGVs. We use this dataset to train the 3D pedestrian detector and evaluate the performance of the pedestrian detectors and tracker. Evaluation of the proposed framework demonstrated the ability to robustly detect and track multiple pedestrians up to a distance of 10 meters. We open-sourced our framework at: https://github.com/bbrito/amr_navigation.

Brachytherapy (BT) is an essential component in the treatment of cervical cancer as it allows for locally delivering a high dose to the tumour with minimal trauma to surrounding tissues and organs at risk (OARs). However, in advanced cervical cancer patients commercially available BT applicators are particularly ill-adapted and therefore result in suboptimal local control and frequent occurrence of substantial tissue morbidity. Additionally, cervical cancer BT is associated with large dosimetric uncertainty which has been shown to significantly impact the delivered dose and the occurrence of normal tissue complications. The clinical outcomes of treatment may be improved through combined efforts in sophisticated applicator design and robust treatment optimisation. Patient-tailored BT applicators have been introduced to improve dose conformity, but currently rely on manual indication of needle channels. Automated needle channel planning software exists, but does not account for OAR dose constraints or uncertainty in planning. Robust treatment optimisation potentially improves the dose conformity of plans in the presence of uncertainty, but relies on the questionable presumption that optimisation of dwell times can fully correct for suboptimal dwell positions. In this thesis, the freedom of source placement that 3D-printed applicators allow and the principle of robust treatment optimisation are leveraged to develop robust needle channel planning software for personalised applicators. This thesis was accordingly divided into three parts. In the first part, literature was reviewed to establish the dosimetric uncertainty budget and evaluate geometric uncertainty of OARs. Inter and intra-fraction uncertainty are likely the greatest contributors to the uncertainty budget, possibly increasing the delivered dose to OARs with up to 4.0±20% (k = 1). Using dose-response models it was established that this may realistically increase the occurrence of moderate to severe morbidity of the bladder or rectum by 1.5 and 3.7% respectively. The BT needle channel planning problem under uncertainty was accordingly defined as the problem of computing multiple feasible, non-intersecting curvature- constrained channels under probabilistic or bounded spatial uncertainty of OARs. In the second part, a tool termed motion-planning quality function deployment (MP-QFD) was developed to select a suitable motion planning class. Using the results from a pilot study among nine medical specialists, this tool substantiated the preferred choice for an incremental sampling-based motion planning algorithm. In the last part, robust variants of sampling-based planners were introduced that are capable of computing trajectories for non-holonomic agents in environments under uncertainty. In a two-dimensional simulated patient case, it was shown that these planners were able to generate near-optimal trajectories that (probabilistically) guaranteed not exceeding OAR dose constraints. Subsequent dose-based optimisation showed that (robust) trajectory planning could theoretically yield treatment plans with improved dose conformity over those generated for conventional applicators. Due to modelling assumptions, robust motion planning did not result in improved dose conformity over a nominal motion planning approach in a worst-case scenario. Future work should therefore focus on improving our understanding of OAR movement in and during BT treatment and validating this theoretical work in a patient case series.