Bimanual coordination is essential for the performance of daily activities, but the underlying motor control mechanisms are not yet fully understood. The goal of the present study is to identify the contribution of contralateral responses in the wrist joints to the performance of a bimanual task. Contralateral responses could possibly be used in rehabilitation therapy to activate hand functions that are affected by a neuromuscular medical condition. In our experiment, participants had to balance a tray in a virtual environment, while either the left or right hand was perturbed. Two identical robotic wrist manipulators intermittently applied force perturbations in flexion or extension direction. Following a perturbation, contralateral responses were present and operated towards stabilization of the tray, for example by allowing an overall faster correction of the perturbation. Notably, flexion perturbations resulted in much larger contralateral responses than extension perturbations. Contralateral responses occurred mainly in the time window for voluntary responses. Results were consistent with our hypothesis for discrete bimanual movements based on optimal feedback control theory: when both hands share one goal, functional coupling occurs in the wrist joints. ... Read more

Safe and reliable autonomous inspection tasks using \ac{MAVs} in cluttered environments are challenging due uncertainties encountered during inspections. In general, algorithms consist of pre-planning paths and executing them. These precomputed inspection paths do not consider potential occlusion by obstacles. In turn, this could render the path infeasible and result in an incomplete inspection of the object to inspect. In order to solve this, a robust method is required, defined as the mitigation of viewing disturbances. This thesis presents an on-line inspection method for occluded environments with static obstacles. The proposed method splits an off-line computed global inspection path into segments and treats each segment as a separate inspection problem. By using an information-based cost function, an \ac{MPC} allows for robust on-line inspection of each of these segments by rejecting obstacle occlusion and collision, if necessary. Additionally, the cost function is designed to be submodular, a mathematical property describing diminishing returns and allowing greedy optimisation while obtaining performance guarantees. Properties of the method are demonstrated in two different environments: with and without obstacles. Due to many sigmoid functions within the cost function, it is a complex optimisation problem. For this reason, scalability is tested and measured in amount of triangles to determine feasible-sized inspection segments. It is shown that for $16$ triangles with one obstacle (both of arbitrary sizes), the calculation time for each \acs{MPC} iteration starts to exceed $15Hz$, becoming more unstable with each added triangle. This effect can be mitigated by discarding information cost of the intermediate cost function. Finally, it is shown that, where the global inspection path is partly occluded, the proposed method handles occlusion by obstacles during inspection at the cost of increased duration of the inspection, while maintaining quality of the solution. However, the predicted performance guarantee by submodularity does not always hold in practice. Future work can explore the integration of sensor measurements to the method or use learning-based approaches to reduce the required on-line computational resources. ... Read more

In modern society cars are one of the most important means of transportation. Unfortunately, many people die in car accidents around the world. Research shows that the number of fatal casualties in car accidents has been increasing for the past decade and that the largest cause of these accidents is the human driver. For this reason, research on fully autonomous vehicles has gained a lot of attention. However, currently autonomous driving is only implemented to reduce the errors of human drivers. More research is necessary in order for fully autonomous vehicles to be implemented and to remove the human driver completely. A robust navigation algorithm which is able to run in real time is one of the challenges in development of fully autonomous vehicles. Important topics in navigation of autonomous vehicles include the path planner and the motion controller. The path planner finds a path for the vehicle from its current location to the target location. At the same time the path planner avoids obstacles and fulfills the non-holonomic constraints of the autonomous vehicle. The motion controller tries to follow the path the path planner made as close as possible by controlling the vehicle. These two topics influence each other and are therefore dependent. In literature little research is done on integrated algorithms that combine path planning and motion control. Therefore, this thesis will research navigation of autonomous vehicles by using an integrated algorithm that includes both path planning and motion control. The objective of this thesis is to develop a Lyapunov stable control algorithm that is capable of planning a path for all possible vehicle maneuvers. Besides path planning the proposed algorithm must be capable of controlling the vehicle along this path. Furthermore, the algorithm needs to include obstacles and the non-holonomic dynamics of an autonomous vehicle. The main contribution of this thesis is an integrated path planner and motion controller for navigation of autonomous vehicles. The stability of the proposed algorithm is proven by using the Lyapunov method. Simulation results prove that the algorithm is capable of planning the path and the motion of the autonomous vehicle with non-holonomic constraints and with the presence of obstacles. ... Read more

This thesis tests the hypothesis that distributional deep reinforcement learning (RL) algorithms get an increased performance over expectation based deep RL because of the regularizing effect of fitting a more complex model. This hypothesis was tested by comparing two variations of the distributional QR-DQN algorithm combined with prioritized experience replay. The first variation, called QR-W, prioritizes learning the return distributions. The second one, QR-TD, prioritizes learning the Q-Values. These algorithms were be tested with a range of network architectures. From too large architectures which are prone to overfitting, to smaller ones prone to underfitting. To verify the findings the experiment was done in two environments. As hypothesised, QR-W performed better on the networks prone to overfitting, and QR-TD performed better on networks prone to underfitting. This suggests that fitting distributions has a regularizing effect, which at least partially explains the performance of distributional algorithms. To compare QR-TD and QR-W to conventional benchmarks from literature they were tested in the Enduro environment from the arcade learning environment proposed by Bellemare. QR-W outperformed the state-of-the-art algorithms IQN and Rainbow in a quarter of the training time. ... Read more

In thedynamic world we live in, the transportation of people and goods in a reliable,efficient and timely manner has grown to be more important than ever. Roads andcities are becoming more congested and the impact of greenhouse gasses canalready be observed. The need for controlling transportation systems, andspecifically fleets of vehicles, more efficiently is therefore now higher thanever. Few methods exist in the literature which utilise historical data toincrease the efficiency of dynamic fleets of vehicles. This work thereforeproposes a novel anticipatory insertion method which incorporates a set ofpredicted requests to beneficially adjust the routes of a fleet of vehicles, inreal-time. This set of predicted requests is derived, in advance, fromhistorical data by clustering comparable requests and predicting similarrequests when assumed patterns in their occurrence are present. This method iscombined with a developed dynamic vehicle routing solver which makes use of arange of heuristics and adaptive large neighbourhood search. The proposedmethod is evaluated using numerical simulations on a range of real-worldproblem instances with up to 1.655 requests per day. These instances representdynamic multi-depot capacitated pickup and deliver vehicle routing problemswith time windows. The method is compared with several other approaches and inorder to quantify the added value of making use of historical data, the methodis benchmarked against a comparable reactive approach which also makes use ofadaptive large neighbourhood search. It is shown that, by making use of theproposed method, on average, 4,58% less distance is required to be travelled bya fleet vehicles while additionally 3,35% fewer vehicles are required to fulfilthe same set of requests. ... Read more

This thesis report is focused on the design of a guidance and control system that is able to minimize the deviation from the desired impact point for a firing range of 1 kilometer, given a 30mm spinning gun-launched projectile with a novel actuator design. This actuator is fixed on the projectile and offers a single force that can only be switched on or off.

In order to test the effectiveness of the guidance, navigation and control (GNC) system, an accurate model for both the projectile and the proposed actuator are constructed. These models will serve as a substitute for testing with working prototypes. The model for the projectile dynamics is constructed based on existing nonlinear 6-DOF rigid-body models that are widely-used and validated. A new, simplified second order model that approximates the dynamics of the actuator, based on available measurements, is constructed.

The overall structure of the GNC loop is defined, with the guidance method of choice being the pre-calculation of a reference trajectory towards the intended target. This method serves to alleviate under-actuation and computation time problems. One of the most important contributions of the thesis is made with the description of a method for transforming the input from a single binary input signal towards two continuous virtual input forces. This method uses a discretization of the rolling motion of the projectile, such that an optimization can be done which results in a transformation from the binary on/off signal to two virtual force inputs. These two virtual force represent the steering forces in the directions perpendicular to the forwards motion that would have the same effect as the binary on/off signal if directly applied on the projectile.

The restructuring of the input allows for the use of PD-controllers to track the pre-defined trajectory. Using a grid-search method, the controller gains are found that best satisfy the design goal of minimizing the dispersion, which is defined by the sum of the mean error and the standard deviation of the impact points.

Analysis of the integrated solution shows that the proposed solution with the input transformation and the PD-controller is able to significantly reduce the projectile dispersion from standard deviations of about 50 cm to just a few cm. The exact performance depends on the frequency of the actuator and the spin rate of the projectile. The best performance is reached with high spin rates and actuator frequencies high enough to match these spin rates. ... Read more

The startup company Fleet Cleaner has developed a mobile robot, specialized in the hull cleaning of large cargo vessels. Navigation and localization of this robot is currently performed manually. This is a difficult process that is greatly complicated during operation. This is mainly due to the availability of relative positioning sensors only, which are prone to error build-up and noise, and to the difficulty of interpreting optical underwater images in turbid water conditions. Instead, operators must rely on acoustic images from a forward-looking sonar. In the field of mobile robotics, Simultaneous Localization and Mapping (SLAM) is an often used technique to improve navigation and localization by utilizing visual information. The objective of this thesis is to develop a sonar-based SLAM framework, tailored to working environment of the Fleet Cleaner robot. The thesis scope has been restricted to the conceptual design of such a framework and the implementation of one of the subsystems, visual odometry. 
A conceptual design of a SLAM system is proposed using a systematic approach. Different working principles are evaluated according to operating conditions and requirements that specify desired behavior. Analysis of operating conditions reveal the limitations of sonar imagery, such as a high signal-to-noise ratio and inhomogeneous intensity patterns. In addition, the environment is sparse, with few distinct recognizable landmarks, limiting feature-based approaches. Because of these limitations, visual odometry is essential to reduce error build-up between loop closure corrections.
A Fourier-based approach to visual odometry is implemented, taking the whole image view into account instead of extracted features. By analyzing the dominant peak in the phase correlation matrix, the in-plane sonar motion between consecutive image frames can be estimated. Several image processing steps are necessary to improve peak sharpness, increasing the quality of registration.
To validate the proposed method, an experiment was conducted during cleaning of the Pioneering Spirit, the world’s largest construction vessel. Under normal circumstances, visual odometry showed less error build-up in the position estimate than wheel odometry. However, outliers appear when driving near the waterline, caused by reflections and wave reverberations. Ultimately, the proposed visual odometry method improves the current positioning system and serves as a basis for an integral SLAM implementation.
... Read more

Knowledge on adversaries during military missions at sea heavily influences decision making, making identification of unknown vessels an important task. Identification of surrounding vessels based on visual data offers an alternative to AIS information (Automatic Identification System), the current standard in vessel identification, which can be spoofed. One visual approach employs human expertise and manually identifies vessels guided by a ship catalog. In order to minimize or potentially eliminate human error and performance limitations, there is strong interest in developing an automated vessel classification pipeline. One such pipeline is currently being developed at TNO, capable of classifying over 500 separate classes. A crucial part of the classification pipeline is retrieving an accurate contour of a vessel from a digital image.
To address this important challenge, this thesis proposes an advanced deep learning pipeline to automatically segment the vessel image into background (e.g. sky and sea) and the object of interest (a vessel). Deep learning models based on Fully Convolutional Neural Networks (FCNs) have achieved high performance on the task of semantic segmentation. Several networks such as CRF-RNN, PSPNet, DeepLab and Mask R-CNN are employed to determine a baseline performance. We will focus on identifying the cause of poor or failing segmentations and aim to construct a robust network capable of handling these challenges. By sampling disturbances, caused by ship distance and camera noise, augmented data sets are built to tune networks to input from on-site images. Additionally, experiments are done to evaluate the influence of different levels of disturbances.
Previous approaches implementing the CRF-RNN network achieved top 1 and top 5 classification accuracies of 31.1% and 44.0% respectively. Employing the DeepLab network, trained to convergence on artificial noise augmented data, we report top 1 and top 5 accuracy of 68.9% and 88.8% respectively. Additionally, implementing an ensemble of classifiers, performance is increased to 73.0% and 91.7% for top 1 and top 5 accuracy respectively. This best result is comparable to the classification results with human annotated ship silhouettes. The human performance accuracy is 73.4% on top 1, and 91.3% on top 5 classification performance. Finally, we show that training on a collection of different levels of image disturbances results in a network that is robust against increasing disturbance in images, while retaining performance on clean images.
... Read more

This MSc. thesis explores the design and implementation of a motion planner for non-holonomic autonomous vehicles in parking spaces. The planner must avoid collisions with static obstacles, satisfy performance, comfort and safety constraints for the motion, satisfy the non-holonomic constraints of the vehicle model, consider the dynamics of the vehicle actuators and be fast enough for real-time implementation.

The motion is planned by solving an Optimal Control Problem (OCP) that is discretized in time in order to obtain a non-linear, non-convex, multi-variable optimization problem. The thesis addresses how to solve this optimization problem so that it results in motions that satisfy the requirements. Specifically, the motion is split into a number of waypoint tracking sections, planned by correspondingly simplified optimization problems. The thesis also addresses methods to further simplify the optimization process in order to reduce the implementation time.

The results show the planner satisfies the constraints but does not quite work in real time. Recommendations for improving the planner in the future are given in the report. ... Read more

Learning capabilities are a key requisite for an autonomous agent operating in dynamically changing and complex environments, where pre-programming is not anymore possible. Furthermore, it is essential to guarantee that the learning agent will act safely by considering its stability properties. In this thesis, novel conditions are proposed, aiming to examine stability of the learned dynamics for two important model classes; namely Rectified Linear Unit (ReLU) Deep Neural Networks (DNNs) and Locally Weighted Learning (LWL). For the former method, a theoretical and computational framework is developed by establishing an equivalence between ReLU DNN models and Piecewise Affine (PWA) systems. This allows to leverage well-known tools of PWA system analysis, and consequently compute, characterize equilibria and determine their region of attraction for ReLU DNNs. Due to their increased complexity, a structured search for appropriate stability conditions was performed for LWL methods until the optimal trade-off between conservativeness and computational efficiency was obtained. These stability conditions are given as Linear Matrix Inequality (LMI) problems and they consist the first stability results in literature for these two model classes. Their efficacy is assessed in numerical and real-world dynamical systems and it is shown that the proposed LMIs are not unreasonably conservative, as they can evaluate accurately the stability properties of these two representations. Finally, this work demonstrates how to formulate appropriate stability conditions for learning methods in a principled manner.
... Read more

Automated vehicle control provides advantages in transport efficiency, redundancy, human-safety, flexibility, and parallelism. Extensive research has been dedicated to direct-following lateral control methods, using a single preview point as reference signal. However, these methods give rise to significant tracking errors. As alternative to a single point, a continuous path can be used as reference signal for vehicle control. Using a continuous reference path, accurate and comfortable control actions can be computed, while avoiding reference tracking errors. Therefore, robust reference path generation is an essential part of lateral vehicle control.

The goal of this research is to develop a generic, robust reference path generator for lateral vehicle control. Currently in path generation, the state-of-the-art method is based on repetitive polynomial fitting. This method inherently contains two main weaknesses. Firstly, it is not robust to sensor noise and other real-world disturbances. Secondly, as a result of the repetitive fitting, a discontinuous path is generated. This is undesirable, because it leads to an unfeasible reference path, since vehicles can only produce and track continuous trajectories. Moreover, a discontinuous reference path results in the need for path smoothing when used in comfortable lateral vehicle control. Fundamentally, this smoothing leads to inaccurate control, caused by manipulation of the original, discontinuous reference path. To overcome these weaknesses, a new Model-based Path Generation method is presented.

The development of a new, generic method for robust path generation is the main contribution of this research. This new path generation method is capable of producing feasible vehicle trajectories based on unfeasible waypoints. The performance of this method is evaluated in simulations and experiments, benchmarking it against polynomial fitting path generation. Furthermore, path generation robustness is assessed based on the outcome of a disturbance sensitivity analysis. The results are in accordance with the hypothesis stating that the new method outperforms the benchmark method in terms of path accuracy, robustness, continuity, and general applicability. ... Read more

Neoclassical economics dictates the decision-making process of economic agents as the mathematical problem of maximizing utility over a prescribed planning horizon. The mathematical similarities with optimal control theory lead to a new interpretation of economic agents as optimal controllers. Pontryagin's maximum principle generates the necessary conditions, but the economic consequences become clear when its historical development is followed. It is found that the Euler-Lagrange equations result in a no-arbitrage condition in economics, and Hamilton's canonical equations describe the change in asset allocation and the asset price over time. The Hamiltonian itself is equivalent to the economic surplus of the agent, and the maximum principle requires that it is maximized along the optimal trajectory with respect to the control actions. This gives a different, myopic perspective to the economic agent, being an agent that maximizes economic surplus instantaneously instead of utility over an entire planning period. ... Read more