Airlab, a collaboration between TU Delft and Ahold Delhaize, is developing Albert, a robot tailored to work in a complex supermarket environment. Key to Albert is a product detection and classification module that tells it what products to grasp and where they are located in a shelf. Albert’s existing YOLO‑based product detector a significant issue: Adding new products without re‑training the whole model is impossible. Especially in a dynamic supermarket environment with an ever‑changing stock, the latter is a major issue.

This problem will be the main focus of this paper and is addressed through few‑shot learning, which predicts similarity between query and target products. This simplifies adding new products to just supplying new target images. Few‑shot learning also requires significantly less data to train on. In supermarkets with 300.000 different products, requiring only a few images per product is a major advantage. For this reason, this paper aims to deploy a few‑shot model to classify products as either the target class or non‑target class for Albert’s picking task and defines the following research question: “What few‑shot classifier can identify products in a supermarket environment, is able to detect non‑target classes, and meets the requirements of deployment on a robotic platform like Albert best?“

This paper first analyses the potential of using TRIDENT and P>M>F, two state‑of‑the‑art few‑shot models, for deployment on Albert, and evaluates them on the requirements of this paper. P>M>F performs better on all requirements, which makes it the preferred model for Albert. However to work well, it still requires adjustments. Its inference time is still too high to work on Albert and it cannot classify query images as not the target product.

For this reason, this paper uses P>M>F’s two key ideas to construct Product‑ProtoNet, a new Albert‑suitable few‑shot model: 1) Using a good pre‑trained feature extractor; and 2) Comparing query images to a set of classes and matching only to the likeliest. P>M>F uses a ProtoNet model for classification that essentially does this; Like ProtoNet, Product‑ProtoNet constructs class prototypes from one or multiple examples of class images. Product‑ProtoNet then uses a sigmoid classifier to predict if query images have the same class as those prototypes. It compares query images to a set of similar class prototypes(helper prototypes) and classifies it as the likeliest. Product‑ProtoNet uses a ViT pre‑trained with DINO to extract image features. To bring down inference time, Product‑ProtoNet computes product prototypes before deployment.

With an accuracy of 99.1% on product classes seen during training and 99.8% on novel classes in a realistic supermarket setting, a low inference time of 2.89 ms and a memory usage lower than 4GB, Product‑ProtoNet is the only model that passes all requirements of this paper. When deployed on Albert Product‑ProtoNet successfully guides Albert to the right product in 97% of attempts. This makes Product‑ProtoNet the only few‑shot classifier that can identify products in a supermarket environment, is able to detect non‑target classes, and meets the requirements of deployment on a robotic platform like Albert.

Spatial object detection and environmental understanding are fundamental aspects of autonomous driving in mobile robots. In this work, a stochastic approach to 2D LiDAR and stereo camera sensor fusion for object detection is presented. By tracking LiDAR clusters 360° around the robot and continuously updating knowledge of their corresponding object types using Bayesian inference when they are within the camera’s view, uncertainties in the detection algorithm are addressed. Additionally, localisation and sensor uncertainties are accounted for by using Gaussian distributions to model the locations of entities. By estimating the state of these entities, dynamic objects can be tracked throughout the environment, while remembering their type from when they were last visible in the camera’s view. The results demonstrate correct linking of camera detections with LiDAR clusters, with an average positional inaccuracy of objects of 0.12m in simulation, and 0.25m across different experiments on the robot, as well as the ability to track humans even when they move out of the camera’s field of view with an average tracking error of 0.33m in simulation.

This thesis investigates the stability and robustness of the Lyapunov Actor-Critic (LAC) algorithm in comparison to the widely-used Soft Actor-Critic (SAC) algorithm. Motivated by the need for reliable and robust control systems capable of operating in dynamic and unpredictable environments, the research initially aimed to explore LAC's performance in an effort-controlled robotic manipulation task. However, the focus shifted to a reproduction study of Han et al. 2020 after identifying key discrepancies in the implementation details, hyperparameters, and undocumented aspects of their study.

The primary goal was to validate the reproducibility of Han et al. 2020's results and assess the role of critical parameters, such as the α₃ stability constraint, in the performance and stability of the LAC algorithm across various simulated environments. By reimplementing and rigorously testing the algorithms, our study confirmed that the α₃ parameter significantly impacts the performance and stability of the LAC algorithm, particularly in complex environments where maintaining high stability is essential for successful operation. The results demonstrated that a more conservative α₃ value, which tightens the stability constraint, improves performance and stability but also increases the time needed for the agent to adapt and stabilize its policy, highlighting the need to balance stability and adaptability in scenarios demanding both high performance and strict stability. Despite some small differences, the study confirmed the reproducibility of Han et al. 2020's findings, demonstrating the potential benefits of the LAC algorithm compared to SAC in terms of stability and robustness. Furthermore, the study showed that other hyperparameters, such as network architecture, training length, and horizon length, greatly influence the performance and stability of the LAC and SAC algorithms, underscoring the importance of precise hyperparameter tuning, transparent reporting, and consistent experimental configurations in reproducing and validating results in reinforcement learning research.

By validating the stability of the LAC algorithm across different simulated environments, and through the development of detailed guidelines for key hyperparameters and a robust codebase, this research provides a solid foundation for extending the algorithm's application to more complex tasks, such as effort-controlled environments. These contributions facilitate reproducibility and pave the way for future research into the LAC algorithm's ability to operate in dynamic and unpredictable environments, ultimately supporting the safe deployment of learning-based controllers in real-world systems where stability and reliability are paramount.

These days, people see more and more applications for drones, including monitoring rainforests to protect plant and animal species. However, drones face challenges when navigating through the dense and cluttered vegetation of the forest. These environments necessitate advanced autonomous detection and navigation to make the drone traverse robustly and fly safely. In addition, the forest brings extra challenges, such as blocked signals for GPS localisation, remote control, and remote supervising.

In this thesis project, a drone is designed, built, and programmed to navigate autonomously in the rainforest with complete onboard computing and no GPS localisation. This 500-gram drone is being extensively tested and optimized in real forest conditions, and a dataset is being created from its autonomous flights to simulate various configurations of the path-planning algorithm. The results of these simulations on this dataset are then used for thorough research on how the algorithm can downscale to smaller systems and how this affects performance.

By using the results of this research on downscaling, a 100-gram drone is built and programmed to fly in forest conditions with complete onboard computation. Challenging on this small-size drone is the use of low-quality lightweight sensors and processor. The processor only weighs 10 grams, and the depth camera weighs 8 grams. Unique on this small drone is the 3D path planning fully computed onboard and the implementation of a new type of depth camera.

As robotics will play a crucial role in the future of our modern societies, the need for advancements in the field is more pronounced than ever. While robots are already present in industrial settings, they are noticeably absent from dynamic environments. Dynamic environments are characterized by other moving agents, such as humans, varying tasks and high safety requirements. With the aim to deploy robots in such environments, Trajectory Generation (TG) becomes crucial. TG approaches aim to compute sequences of control commands that bring the robot from its current configuration to a desired goal state while avoiding collisions with obstacles and itself. Thus, it is directly placed between task planning, the problem of defining what high level tasks should be executed in which order, and control, the problem of executing motor commands. A good solution to TG must be fast, to cope with changes in the environment, flexible to different goal definitions, and it should promote safety. Most advancements in the field of robotics in TG focus either on manipulators or on mobile robots. However, the combination of both systems seems inevitable for deployment in human-shared environments.

TG for mobile manipulation is usually formulated as an optimization problem of a finite time horizon. This approach is known as Model Predictive Control (MPC) and is widely used in the field of autonomous driving thanks to its feasibility and stability guarantees. In Chapter 4, we present a method to bring MPC to mobile manipulation. The method formulates the TG problem for the entire kinematic chain and relies on Free Space Decomposition (FSD) for collision avoidance. This leads to reasonable control frequencies of 10Hz independent on the amount of collision obstacles in the environment. Importantly, this approach allows for coupled motion of the mobile base and the manipulator. This is beneficial in situations where synchronization of the two subsystems is crucial, such as opening doors or moving obstacles around. Despite these simplifications on the environment representations, computational costs limit the applicability of MPC to mobile manipulation as motion is not considered truly reactive and different components, such as goal attraction and collision avoidance, are not easily separable.

A recent novel approach to receding horizon control is the formulation as a purely geometric problem. Early successes in this direction, including Cartesian Impedance Control (IC) and Artificial Potential Fields (APF), led to the formulation as sets of dynamical systems on smooth manifolds in the configuration space. The framework of Optimization Fabrics (fabrics) unifies such ideas, offers stability guarantees in static environments, and results in highly reactive behavior similar to simple low-level controllers. This framework relies on non-Riemannian geometry to shape a smooth manifold of the configuration space with individual behaviors, such as collision avoidance, joint-limit avoidance, and goal attraction. In Chapter 5, we present a generalization of fabrics to dynamic environments. We refer to the resulting framework as Dynamic Fabrics (DF). The generalization uses time-parameterized manifolds to integrate moving obstacles and time-parameterized reference trajectories. The latter is especially important for long-horizon TG that may exhibit local minima. Importantly, Chapter 5 shows that the dynamic component of DF is required when coping with moving obstacles. As repulsive forces in fabrics are proportional to the approaching speed of obstacle and robot, collision avoidance in a pseudo-static fashion is not sufficient when the robot is moving slowly. Finally, we deploy the general framework of DF to several real-world settings showing the applicability of the framework to mobile manipulation. First, we present a way to integrate non-holonomic constraints into the framework. Despite loosing formal guarantees on convergence, the method is shown to be the natural extension to wheeled mobile robots characterized by non-holonomic constraints. Second, Chapter 6 presents a symbolic implementation of fabrics to achieve higher control frequencies. Symbolic implementations are possible because the framework of fabrics is based on differential equations of second order, for which a closed-form solution exists. For changing environments, obstacles states are then only concretized at runtime. Additionally, we show symbolic hyperparameters can be tuned automatically to achieve expert-level tuning performance. Third, to overcome high requirements on the perception pipeline, Chapter 7 integrates different implicit environment representations into the framework. Using Signed Distance Field (SDF) and FSD for example is widely used in mobile robotics when formulating TG as MPC. We show that the same representations can be used in fabrics while achieving faster solver times. Finally, Chapter 8 deploys a mobile manipulator controlled by fabrics in a supermarket. Dexterous manipulation is programmed using learning-from-demonstration, with fabrics as the underlying encoding. That allows to teach rather than program complicated behaviors while maintaining properties on collision avoidance.

This thesis presents insights into aspects of motion planning, advances the framework of fabrics for TG, and compares it extensively to the more commonly used method of MPC. Through the ideas presented in this thesis, we hope to encourage the usage of geometric properties of robotic systems deployed to human-shared environments. This approach does not only provide reactive TG but also may act as a compact encoding of trajectories for learning-based methods in the future.

Object pushing in robotics has numerous applications, but it often relies on room-bound object tracking systems such as Motion Capture (MoCap) for accurate object pose acquisition. Such systems limit the potential use scenarios, since they add complexity and cost and require expansion of the sensor infrastructure for expanding the operational areas. To address these limitations, we propose a framework that integrates object pushing and vision-guided object pose estimation using a monocular RGB camera that is placed on the pushing agent. By leveraging the temporal coherence of the scene, we accelerate the 2D object detection stage that is common in the field of object pose estimation. Our contributions include the development of a novel framework that continuously pushes an object while simultaneously tracking its pose. We incorporate dynamics by efficiently using a Model Predictive Path Integral controller’s predictive model to predict the object location in the forthcoming image, resulting in an extreme reduction of computational cost for 2D object detection, without sacrificing the accuracy of the pose estimation. Our framework achieves real-time object pushing with accurate object pose estimation, which we demonstrate with real-world experiments. Furthermore, we provide a novel object pose estimation dataset in the widely used BOP format, specifically for robot planar pushing with an on-board camera. Our approach pushes towards eliminating the need for room-bound sensor systems, expanding the potential use cases. By considering temporal coherence and scene dynamics, our framework challenges recent object pose estimation methods that fully process each image as uncorrelated individuals, while providing a promising solution for real-time object pushing.

Automated vehicles represent an exciting advancement in transportation, offering a range of benefits that have the potential to revolutionize how we travel. They can improve safety, efficiency, accessibility, and sustainability, holding promise for transforming our cities and communities. However, generating safe, comfortable, and efficient motion plans, especially in interactive scenarios like forced lane merging, remains a significant challenge in the field.

Lane merging is a pivotal skill for automated vehicles, as it frequently involves changing lanes to reach a destination. For instance, when approaching an intersection, a vehicle might need to merge into a specific lane beforehand to execute a turn later. Traditional pipelines in automated driving typically decouple prediction and planning. They assume perfect upstream prediction and generate robust motion plans to avoid collisions with multi-modal predictions. However, in dense traffic conditions, conservative planning might hinder the ego vehicle from merging effectively, resulting in it becoming stuck. This underscores the necessity of combining prediction and planning, a concept we term interaction-aware planning algorithms.

The first major contribution of this thesis is an efficient game-theoretic behavior planner that captures interactions under different behavior modes. In this approach, we represent the behaviors of the vehicles as actions in a matrix game and select the Nash equilibrium to capture their mutual influence. To generate the cost of the action pairs, we model the merging process as a gap selection process and evaluate the trajectories generated by interactive models. The effectiveness of the proposed planner is validated in the high-fidelity CARLA simulator.

In the real world, human drivers may not always adhere rationally to the equilibrium of a game model. They could choose a behavior mode different from the game theory solution. Therefore, it might be more beneficial to consider different motion modes simultaneously, rather than favoring the "most likely" one while neglecting the others.

For this purpose, we also explore the usage of Branch Model Predictive Control (B-MPC) in this thesis. By predicting the motion of the surrounding vehicle as a scenario tree, the B-MPC approach can generate a trajectory tree as a motion plan. By executing only the root node, the ego vehicle can consider different future scenarios simultaneously and plan contingency motions. We further extend the B-MPC approach by incorporating interactive policies, using different solving schemes, and including collision avoidance constraints that consider the orientations of the vehicles. The effectiveness of these proposed methods was validated through experiments conducted in a handcrafted lightweight simulator.

Overall, this thesis focuses on developing interaction-aware planning methods to facilitate safe, successful, and comfortable lane merging scenarios. Nevertheless, the major limitations lie in the modeling error and the potential long-tail issues. To address these challenges and further improve motion planning, future work could explore data-driven (learning-based) approaches that leverage real-world driver behavior to generate more informed and adaptable motion plans.

Visual inspection of liquid medicine containers for contamination and defects is mandatory and crucial to ensure their safety for injection. This document presents research and development of three modules of the Visual Particle Inspection Subsystem (VPIS), an automatic inspection subsystem with the task of detecting and classifying particle contamination in liquid medicine containers. The proposed VPIS comprises three modules for which research and development is performed separately: the backgrounds subtraction and segmentation module, the classification module, and the tracking module.

For the background subtraction and segmentation module, a solution is proposed with filtered temporal background modelling and locally adaptive threshold segmentation. This solution works on the assumption that moving objects such as particles and bubbles are sparse and do not occur more than twice in a pixel in twenty frames. Therefore, a representative incomplete background cluster can be obtained by removing the two lowest or highest pixel values depending on the mode of illumination. Next, the average value of the background cluster is used for background subtraction and the standard deviation of the background cluster is used to determine a locally adaptive segmentation threshold. The solution has been positively evaluated for detection of low contrast objects, insensitivity to disturbances, processing time, and parameter configuration. Additionally, an add-on solution is proposed that makes it possible to detect objects in the challenging region near the rubber stopper of a syringe.

For the classification module, out of four candidate classification methods a Convolutional Neural Network (CNN) is chosen for the classification of single object detections. The CNN classifier achieves an accuracy of 0.93 and is used as a baseline classifier for the rest of this research. Next, with three exploratory research questions, research is performed into opportunities and pitfalls for this classification problem summarised below.
-It is found that certain handcrafted features correlate to the classification accuracy of a detection. A method is proposed to predict the classification accuracy of detections and filter out detections that cannot be classified reliably. Compared simple filters that filter out detections based on a single feature such as area or total contrast, the proposed method should achieve a higher subsequent classification accuracy and reject less detections.
-With the simplest classification strategy that is often used for inspection, a container will be rejected if a single detection is classified as a particle. It is shown through simulation that this classification strategy is not effective for this classification problem. This is as the large number of bubble detections in a clean container would result in a high false container rejection rate.
-Two multi-detection classification strategies are proposed that use multiple detections for classification. Median voting classification uses results from the tracking module to classify multiple detections of the same object. Multi-positive classification does not use tracking results but requires multiple detections to be classified as particles. Simulation results indicate that with both strategies, the classification accuracy for containers can be drastically improved. It is theorized that the best results can be achieved with median voting classification...

Simulation environments are useful for a wide range of applications and their functionalities continue to improve every year. The aim of this thesis project was to create a simulation environment with high levels of realism and assess its capabilities through the use case of generating distributed drone traffic rules.

This thesis project was executed at the company Almende in cooperation with its sub-company DoBots. The distributed drone traffic rule generation use case was a contribution to the ADACORSA project which Almende takes part in. The goals of the thesis were the following. A simulation environment incorporating elements of realism was created using the software tools Almende has experience with. The capabilities of this environment were tested through the said use case. According to company requirements, this process was realised using an incremental complexity approach, and it utilized machine learning techniques. It was expected that the incremental complexity approach would help speed up the learning process of more complex scenarios, while machine learning would provide the benefit of smart automation and impartiality to solutions. For this process, a model was also devised for creating such rules, incorporating the incremental complexity approach and a standard machine learning technique. Finally, the benefits of the incremental complexity approach were also assessed.

The four building blocks of the implemented simulation environment were Gazebo, PX4, ROS2 and the iris drone model. The code was built upon the OpenAI ROS software package, with extensive modifications. The main work done during the implementation consisted of multiple parts. The original software package was cleaned of unnecessary elements, the PX4 software and the iris drone model were integrated into the environment and the simulation speed was increased. The whole setup was migrated from ROS to ROS2. The use of multiple drones in the same environment was also solved. An implementation of the deep Q-learning algorithm, which was selected as the learning method, was added. Different scenarios were created, as well as some logging and visualization tools. Additionally, the system was set up to run in docker containers and was highly parameterized for future use.

Different elementary traffic scenarios were implemented in the simulation and then executed. Based on these the following points were determined. It was concluded that the created simulation environment was well-fitted for running scenarios for traffic rule generation. It was also suitable for executing the scenarios using the incremental complexity approach, although its usability also depended on how the neural networks of the scenarios were set up. The environment was also feasible to work with machine learning from an engineering viewpoint, however, it did not perform well for its scientific research. This was due to not having statistically well-supported results, which was caused by the long time it took to execute a scenario. The benefits of the incremental complexity approach were also examined. For the simple case tackled in this project, it was found that the incremental complexity approach provided no significant advantages with regard to learning speed.

This master thesis introduces Hierarchical Active Inference Control (HAIC) as a control method for nonholonomic systems. This method only requires tuning of a minimal number of hyperparameters and has a relative low computation load. HAIC is based on recent research done in the application of the neuroscientific theory of Active Inference for robot control. The hierarchical aspect of this method introduces two layers of Active Inference Control (AIC) and the introduction of an additional decision hyperparameter. The top layer AIC is designed such that it takes the nonholonomic constraint into account. This causes HAIC to be able to control a nonholonomic mobile robot to a three-dimensional reference state in comparison to regular AIC which is unable to. The selection of the introduced hyperparameter gives a trade-off between robustness against noise present in the measurements and possible error in
the y-dimension of the local reference frame.

Convergence of a two-wheeled differential-drive mobile robot to a reference state using HAIC is demonstrated both by simulation and physical experiment. Simulations are done for a large range of different starting states. These simulations show the ability of HAIC to converge the mobile robot for any starting state. It also confirms that a higher value for the decision parameter causes a larger error. Another group of simulations is done investigating the new
hyperparameter introduced with HAIC. These simulations check the ability to converge the system when different noise levels are present in the measurements. It is concluded that tuning of the newly introduced hyperparameter needs to accommodate the noise present in
the measurements. Where the higher the noise, the higher the value of the hyperparameter is needed for convergence.

A physical experiment is done where the robot is controlled to five different reference states. Data obtained from this experiment shows the ability of HAIC to control a physical nonholonomic system. Like the simulations, it is also shown that a correct value is required to set the newly introduced hyperparameter. Additionally, the physical experiment shows that HAIC is not robust when disturbances can cause sudden large value changes in the measurements.

Autonomous robots are often successfully deployed in controlled environments. Operation in uncontrolled situations remains challenging; it is hypothesized that the detection of abstract discrete states (ADS) can improve operation in these circumstances. ADS are high-level system states that are not directly detectable and influence system dynamics. An example of a typical ADS problem that is used in this thesis is that of a wheeled robot driving through puddles of mud that, when entered, alters the velocity of the robot. When the robot is in such a puddle, it is in an ADS 'mud', and when it is not, it is in an ADS 'free'. ADS can be indirectly inferred through the analysis of lower-level data such as the velocity of the robot.

The goal of this thesis is to design a general abstract discrete state estimator (ADSE) operating with limited prior knowledge. An ADSE is a hierarchical system for detecting changes in ADS. The ADSE should be general; applicable to multiple ADSE problems. The ADSE should further operate under limited prior knowledge: only assuming that the amount of ADS and the ADS that describes the regular operation are known. The basis for the ADSE designed in this thesis is a Gaussian hidden Markov model (GHMM), a hidden Markov model enhanced with Gaussian emissions.

Randomly generated experiments are done on a simple but general ADSE problem. Two unsupervised learning methods derived from Expectation Maximization are evaluated, namely Baum-Welch (BW) and forward extraction (FWE). FWE is introduced in this thesis and is a simpler implementation of Viterbi extraction, leveraging assumptions of ADSE to in theory gain computational efficiency. We found that both BW and FWE exhibit superior performance compared to a likelihood-based baseline estimator when the maximum score of the learning curve is considered. When the final score is considered, in some cases, FWE displays a deteriorating learning curve, resulting in worse final scores compared to the baseline. Furthermore, it was found that the lower the overlap coefficient (therefore the less similar the ADS), the higher the maximum reached score. It was further shown that BW exhibits better convergence than FWE to the true model parameters. Besides this, FWE obtained comparable or in some cases even superior scores compared to BW.

In general, from the results, the diversity of the experiments conducted, and the assumptions made we can conclude that the GHMM can be a general method for an ADSE with limited prior knowledge. To quantify the suitability of the GHMM for ADSE, further research should include the evaluation of different ADSE methods on the same problem. There exists a tradeoff between the lower computational cost FWE and the more stable but more computationally intensive BW learning. Therefore, future research can include a combination of these methods. Other extensions include extending the GHMM to a Gaussian mixture hidden Markov model to allow for the modeling of more complex distributions, or the application to multiple states or a changing environment.

In the literature, neural network compression can significantly reduce the number of floating-point operations (FLOPs) of a neural network with limited accuracy loss. At the same time, it is common to manually design smaller networks instead of using modern compression techniques. This thesis will compare the two approaches for the object detection network YOLOv7. YOLOv7 can run in real time on a desktop GPU. For edge GPUs a smaller version, called YOLOv7-tiny, was manually designed by the authors of YOLOv7. This thesis answers the question: Can a state-of-the-art compression of YOLOv7 achieve higher accuracy than YOLOv7-tiny at the same number of floating-point operations?
First, two state-of-the-art compression methods are selected and compared on YOLOv7-tiny. Then the best performing method, GBIP, is used to compress YOLOv7 till it has the same number of FLOPs as YOLOv7-tiny. From the experiments it is determined that GBIP is not able to achieve higher accuracy than YOLOv7-tiny at the same number of FLOPs.