With the need for robots to operate autonomously increasing more and more, the research field of motion planning is becoming more active. Usually planning is done in configuration space, which often leads to non feasible solutions for highly dynamical or underactuated systems. With kinodynamic planning motion can also be planned for this difficult class of systems. However, due to the difficult nature of the problem, computation time is an issue. RRT CoLearn is a novel variant on the original RRT algorithm that tries to decrease computation time by replacing computational heavy steps in the algorithm with supervised learning. In this thesis the performance of RRT CoLearn is investigated, and it is found that it does not work on multi-DOF systems. Furthermore a novel steering function is presented called Inverse Dynamics Learning, which is shown to converge over five times faster than RRT CoLearn and also converge on a highly non-linear 2-DOF system.

Human-robot collaboration can be improved if the motions of the robot are more legible and predictable. This can be achieved by making the motions more human-like. It is assumed that humans move optimal with respect to a certain objective or cost function. To find this function an inverse optimal control approach is developed. It uses a bilevel optimization for finding what linear weighted combination of physically interpretable cost functions best mimics human point-to-point motions. The upper level compares the optimal result of the lower level with a reference motion. Two depth cameras are combined in a motion capture setup to record this reference motion.

The human arm is modeled as a seven degrees of freedom manipulator, similar to a robot arm model of the YuMi. The bilevel optimization is done with both models, resulting in two differently weighted cost functions. The cost function that was derived using the robot model is then used to generate new motions for the corresponding real robot arm. The results of an experiment show that humans experience these motions as more anthropomorphic and feel at least equally as safe compared to existing motion planning strategies. This proves the validity of the inverse optimal control method and shows it is a step forward for better collaboration between humans and robots.

Kinodynamic motion planning for a robot involves generating a trajectory from a given robot state to goal state while satisfying kinematic and dynamic constraints. Rapidly-exploring Random Trees (RRT) is a sampling-based algorithm that has been widely adopted for this. However, RRT is not fast enough to enable its use in industrial applications. Recently, supervised learning has been used to pre-learn time consuming steps of RRT which resulted in improvement in planning times. The supervised learning models require cost and control input of the system as training data which are generated using optimal control.

The training data can be obtained either by indirect optimal control or direct optimal control techniques. In this thesis, both the techniques are each used to generate cost and control inputs for a two-link manipulator using random initial-final state pairs. Then each dataset is used to train a model and the datasets are compared based on certain training metrics. K-nearest neighbours regression and multi-layer perceptron neural network are the supervised learning models used in this thesis. It is observed that both the datasets result in similar convergence of the models, but indirect optimal control approach allows upto 24-fold faster data generation and upto 3-fold reduction in dimensionality of training data compared to the direct optimal approach.

Real-world robots have torque limits based on actuator configuration. The torque limits are modeled as control constraints in both the optimal control techniques and the effect of
this restriction on data generation and supervised learning is studied in this thesis. Direct
optimal control is found to be better for data generation in this case due to the ease of
applying control bounds as inequality constraints on the function approximations. Indirect
optimal control is very tedious as active constraints should be known a priori to determine
the switching points. An alternate method is explored instead where samples are generated similar to the unconstrained case but samples violating the constraints are removed. Poor control input learning is observed in both approaches and the models struggled to extrapolate. It is hypothesised that this is due to inability of the constrained data to fully capture the system dynamics. However, good cost prediction is achieved using neural networks.

The last decade has marked a rapid and significant growth of the global market of warehouse automation. The biggest challenge lies in the identification and handling of foreign objects. The aim of this research is to investigate whether a usable relation exist between object features such as size or shape, and barcode location, that can be used to robustly identify objects in a bin.

A deep convolutional neural network (CNN) is built in MATLAB and trained on a labeled dataset of thousand product images from various perspectives, to determine on which surface of a product the barcode lies. Training results show that while the training set accuracy reaches 100%, a maximum validation accuracy of only 45% is achieved. A larger dataset is required to reduce overfitting and increase the validation accuracy. When sufficient classification accuracies are reached, smart picking strategies can be implemented to efficiently handle products.

This thesis report proposes a framework to implement Navigation, Guidance and Control (GNC) systems, that enable point-to-point autonomy for displacement vessels. A model-based control approach is chosen as the basis of the GNC systems. The resulting algorithms are implemented for verification in a 1:25 scale model of a Azimuth Stern Drive (ASD) 3111 Damen tug named "Damen Autonomous Ship", aka DASh.

First, a compact maneuvering model that captures relevant dynamics of displacement vessel is formulated and identified using system identification.

Secondly,the guidance system is automated such that it connects an initial state to a goal state with a collision
free path that satisfies all input and differential constraints of the vessel model. To this end, the kinodynamic
Rapidly-exploring Random Tree (RRT) algorithm is extended to use a maneuver automaton and optimal motion primitives in its steering function. A learned cost-to-go distance metric for the state space is formulatedto efficiently calculate distance between states, which is used to search for nearest neighbors in the kino-
dynamic RRT algorithm. The performance of the planner using the learned cost-to-go distance metric is compared to a minimal curve length distance metric based on
Dubins Curves and the commonly used straight-line Euclidean distance metric. It is shown that the learned cost-to-go and the minimal curve length distance metric result in paths of similar performance while the Euclidean metric performs severely worse.

Lastly, the navigation and control systems are implemented on DASh. Due to disturbances present in real
world environments, the paths must be tracked using feedback control. State estimation for navigation,
based on position and heading measurements is performed by implementing an observer using a Extended
Kalman Filter (EKF). Non-linear model predictive control (NMPC) in combination with thrust allocation is used to control the vessel during path execution. Due to real time requirements of DASh, the EKF, NMPC and the thrust allocation algorithm are directly implemented inefficient C ++, on the on-board computer of DASh.
It is shown that NMPC converges to static reference positions faster and with less control inputs compared to
traditional non-linear PD control. It is also shown that time-varying trajectories created by the kinodynamic RRT can be executed successfully. This shows that the identified model is suitable for use in model-based
control and that the planned paths indeed satisfy the input and differential constraints of the vessel.

Currently the prevalence of general purpose mobile robots with manipulation capabilities is still low, despite various applications of such systems such as: disaster response, payload delivery, and assistive/service tasks. A suitable design for such a robot would be that of a torque-controllable quadrupedal manipulator. Its capability for legged locomotion enables high mobility, specifically on rough terrain, while its quadrupedal morphology provides a relatively large and stable base of support compared to bipedal robots. Torque-controlled joints allow for safer and more controllable interaction with the environment.

For such a system a challenge lies in the design of a controller that actually achieves the promising capabilities for locomotion and manipulation that the mechanical system offers. One of the currently most promising control frameworks for this purpose is that of hierarchical inverse dynamics. This real-time whole-body control framework allows for dynamic whole-body motions and compliant interaction with the environment while enforcing a strict priority between the desired control tasks. Although promising results have previously been attained with such controllers, it has not yet been applied for the control of a quadrupedal manipulator. The focus of this thesis is on the implementation of a hierarchical inverse dynamics controller for the control of a simulated quadrupedal manipulator, with a particular focus on the design of prioritized sets of control tasks which generate forms of stationary whole-body manipulation.

First a basic set of prioritized control tasks is presented, which is shown to satisfy the robot's most crucial control requirements. Subsequently three extended sets of control tasks are presented, which result in additional desirable emergent behavior of the robot. The first one of these depends on the inclusion of kinematic joint limit tasks to prevent kinematic singularities and self-collision, and to trigger whole-body reaching motion. Secondly a set of tasks is presented which is focused on utilizing motion of some of the torso's degrees of freedom to optimize the arm's posture according to a posture-related cost function. Thirdly a set of tasks is presented which enables contact force control at the end-effector for force-based manipulation. In addition to this final set of tasks, a higher-level controller is presented which detects external forces acting on the robot and computes a desired shift of the position of the robot's center of mass in order to mitigate the balance-disturbing effects of external forces. All of the presented sets of tasks do not exclude each other and allow to be implemented simultaneously in order to combine the individual benefits that they offer.

The results of this research project show that hierarchical inverse dynamics control can be successfully applied for the control of a simulated torque-controllable quadrupedal manipulator. Moreover, it is shown that well-designed sets of prioritized control tasks allow for emergent whole-body behaviors which exploit the advantages that both the robotic system and the control framework offer. Future work will need to investigate the transferability of these results to a physical robot.

Reinforcement Learning (RL) is a general purpose framework for designing controllers for non-linear systems. It tries to learn a controller (policy) by trial and error. This makes it highly suitable for systems which are difficult to control using conventional control methodologies, such as walking robots. Traditionally, RL has only been applicable to problems with low dimensional state space, but use of Deep Neural Networks as function approximators with RL have shown impressive results for control of high dimensional systems. This approach is known as Deep Reinforcement Learning (DRL).

A major drawback of DRL algorithms is that they generally require a large number of samples and training time, which becomes a challenge when working with real robots. Therefore, most applications of DRL methods have been limited to simulation platforms. Moreover, due to model uncertainties like friction and model inaccuracies in mass, lengths etc., a policy that is trained on a simulation model might not work directly on a real robot.

The objective of the thesis is to apply a DRL algorithm, the Deep Deterministic Policy Gradient (DDPG), for a 2D bipedal robot. The bipedal robot used for analysis is developed by the Delft BioRobotics Lab for Reinforcement Learning purposes and is known as LEO. The DDPG method is applied on a simulated model of LEO and compared with traditional RL methods like SARSA. To overcome the problem of high sample requirement when learning a policy on the real system, an iterative approach is developed in this thesis which learns a difference model and then learns a new policy with this difference model. The difference model captures the mismatch between the real robot and the simulated model.

The approach is tested for two experimental setups in simulation, an inverted pendulum problem and LEO. The difference model is able to learn a policy which is almost optimal compared to training on a real system from scratch, with only \SI{10}{\percent} of the samples required.