A wet clutch is a device that transfers torque between two shafts via a hydraulic mechanism. Wet clutch control is key to achieve smooth and fast clutch engagements. Optimal control of a wet clutch is not trivial because of the complexity of the system due to nonlinearities, hybrid dynamics and changing dynamics over time due to changing temperatures and wear. Nowadays simple parametrized feedforward controllers are used in industry. The parameters of the control signal are tuned by hand and updated over time by an operator to account for the changing dynamics. Several solutions to the wet clutch control problem exist in the literature, however the structure of the solutions is fixed beforehand and they rely on expert knowledge of the system. Another challenging aspect of wet clutch control is model uncertainty, currently robustness of performance is considered by running a finite number of experiments, but no hard guarantees can be given. In this thesis a method is developed to automatically synthesize controllers for a wet clutch which are robust to model uncertainties. The method uses Genetic Programming (GP) to automatically synthesize controllers. Using GP controllers for a wet clutch can synthesize without fixing the structure beforehand and in a multi-objective way. Robustness is considered by optimizing the worst case performance. To be able to give a guarantee on the worst case performance of a controller a method, that is able to formally guarantee a lower bound on the worst case performance using reachability analysis, is developed. This method is to costly to incorporate into GP and instead a use cheap estimation of the worst performance in practice. This method was able to find robust controllers which outperformed a hand tuned baseline controller, but in order to compare this method to other methods in literature, real life experiments are needed.

There are many stages that involve humans handling food objects in the processing chains from farms to stores. For some of these tasks it is desirable to look for a robotic solution to either assist the human or even take over that task, e.g. if it is physically demanding, imposes contamination risks or because of economical considerations. Moreover, recently the COVID-19 pandemic revealed even more vulnerabilities in our food processing chains, when seasonal labourers where blocked at borders and some food processing sites turned out to be "corona hotspots" with the result that whole food processing chains were disturbed. A step towards solving some of these problems is studying robotic grasping, since it is a crucial skill for many manipulation tasks. This work focuses on force closure grasps for pick-and-place tasks. Since humans grasp novel objects effortlessly, a human inspired approach is proposed that combines visual and tactile sensory information and learns from its mistakes thanks to reinforcement learning. Visual features obtained from an RGB-D camera in combination with pressure information from sensors on the finger tips of the robotic hand are used to set the desired grasp force adaptively while holding the object. A novel algorithm is proposed for learning to grasp with minimal force: LIFT (Learning of Initial Force and Tuning). The novel grasp approach is evaluated in a Gazebo simulation environment and on a real-world robotic setup. The approach results in successful grasping in both simulation and on the real-world setup in tens or hundreds of learning interactions, depending on the size of the state-action space. Success rates of 96.3 % and 96.7 % are obtained in simulation and on the real-world setup, respectively. The results from the experiments indicate that the approach is successful in grasping with minimal force.

Robots that use legged locomotion have the ability to overcome obstacles and can negotiate a wide range of difficult terrains, such as encountered in outer-space missions. In many practical scenarios however, their applicability is still limited, mainly due to insufficient speed and efficiency. On the other hand, robots that use wheeled locomotion are fast and efficient, but are generally confined to flat or prepared surfaces. A relatively new approach, to use the advantages of both forms, is the combination of walking and driving technology into Hybrid Walker-Wheeler technology. In this research we will explore the feasibility of applying Hybrid Walker-Wheeler technology to the Zesbenige Robot (ZebRo). ZebRo is a small walking robot with six One-Degree-of-Freedom (DoF) legs and walks with an insect-inspired gait. It is being developed with the intention to go on a mission to the moon. The objective in this thesis is to increase the speed and energy efficiency of the ZebRo on flat surfaces, while maintaining its robustness and walking capability on rough terrain. We will set up the criteria for a new design, explore various options and parameters and choose a concept. We will then do a number of simulations to analyse the properties of a wheel and a leg and to apply these in the design. The final prototype consists of a single module with a wheel, a one-DoF leg and a custom-design coupling which switches torque, from the motor, between the wheel-axle and the leg-axle. This prototype was tested and evaluated on its electrical power consumption and the torque and speed transmitted to the leg-axle and wheel-axle. From these results, we were able to draw a number of conclusions and make recommendations for a ZebRo equipped with Hybrid Walker-Wheeler technology.

A fundamental task of intelligent and autonomous robots is to infer from observations the state of the world. This inference is generally achieved by employing a filter, which consists of a model and filtering law. Learning this model and filtering law from observations is another fundamental part of robotics, and is generally referred to as system identification. Neuroscientist K.J. Friston has developed a relatively novel theory on biologically plausible human brain inference called the Free-Energy Principle. One of the theories within the Free-Energy Principle, namely that Dynamic Expectation Maximization (DEM), has been suggested as a novel method for filtering and system identification. This method is expected to outperform standard Expectation Maximization (EM) in terms of hidden state and parameter estimation in settings where noise is correlated. However, in order for this neuroscientific theory to be properly used for robot inference, two problems must first be solved. The first of these problems is the fact that the theory is defined in the continuous-time domain, whereas data available for system identification is always discrete. In this thesis I will suggest three discrete-time interpretations for DEM-based system identification. The major difference between the three methods is the information that is embedded in the generalized signals: predictions, derivatives and past data. The second problem is that the filtering method corresponding to the Free-Energy Principle depends on data which is not available: the derivative signals of measured in- and outputs. I introduce two fundamentally different solutions to this feasibility issue: a numerical differentiator and a stable filter. Both of these solutions are shown to find an estimate for the unavailable data. However, the former is shown to significantly outperform the latter. Furthermore, the theory described in this thesis is implemented into a novel python toolbox for system identification. This toolbox can be used as a basis for further research and be approved along with it, until at some point it is ready to be used for real applications. Using the toolbox, the DEM-based identification and filtering methods are tested though various numerical simulations and the results are compared with the EM method. Results show that with the implemented settings none of the suggested discrete-time filtering methods outperforms the conventional Kalman filter. The main cause of this inferior performance is shown to be instability in the filtering method. I make some suggestions for overcoming this problem. As a result of the inferior performance, the joint-performance of the suggested DEM-based parameter- and state- estimation methods also proves to be inferior in terms of parameter estimation accuracy. However, results show that the theoretical parameter optima of the Free-Energy as determined from known hidden states are in fact close on the real parameters, and furthermore show to be invariant to noise correlation. This suggests that should the instability issue as some point be solved and a better means to approximate the theoretical optimum be found, the DEM-based methods might in fact outperform EM both in terms of hidden-state and parameter accuracy in settings with correlated noise.

This thesis is a contribution to the research on Active Inference for Robotics. Active Inference is an intricate, intriguing theory from neuroscience, a field in which it has already gained a greater following and popularity. This theory, based on the underlying Free Energy Principle, provides a unified account of perception, action and learning in the biological brain. It has great explanatory power of the function of the biological brain and furthermore it is mathematically well-defined. This property makes the theory suitable for a translation to robotics, in which it can also provide a unified account of action and perception. This is not only elegant, but potentially very powerful too. The research for Active Inference in robotics is young, but the current research already shows that Active Inference indeed has great potential for robotics control. Literature on Active Inference is narrow and complex, and provides a lot of concepts to work with in a translation to robotics control. Once such concept are the generalised coordinates of motion, which are the instantaneous derivatives of a dynamic variable. The incorporation of generalised coordinates, especially in combination with the assumption that the noise encountered in a dynamic environment is coloured, has great potential to be beneficial for both action and perception when it comes to robot control in real environments. Generalised coordinates provide a reference frame for the gradient descent that is applied to provide the action and perception laws, which in a dynamic setting has to `hit a moving target'. Furthermore, in combination with coloured noise the generalised coordinates are advantageous for dealing with such noise. In this thesis, detailed research is provided with regards to the application of generalised coordinates in Active Inference for robotics. Current research for robotics in which Active Inference has been applied doesn't exploit the full potential of generalised coordinates. Therefore, this research aims to explore the constructs necessary to apply generalised coordinates of motion in an on-line Active Inference control loop of an LTI State Space system. A detailed derivation of the generalised precision, which relates generalised coordinates and coloured noise, is provided. A method for obtaining generalised output by means of finite differences is proposed, that constructs generalised coordinates from the on-line data in scenarios in which the environment does not provide the required generalised coordinates naturally. The method is implemented in the simulation of a one degree of freedom SISO LTI State Space scenario which highlights the potential but also the difficulties still faced when applying Active Inference for on-line robotics control. Besides the detailed derivations of some aspects of Active Inference for robotics, open problems are identified and suggested for future research that can potentially yield methods to apply Active Inference in robotics at full capacity, providing a true biologically plausible robot control method.

The Free Energy Principle, which underlies Active Inference (AI), is a way to explain human perception and behaviour. Previous literature has hinted at a relation between AI and Linear-Quadratic Gaussian (LQG) control, the latter being a textbook controller. AI and LQG are, however, defined with different settings in mind: LQG has access to inputs, whereas AI estimates these; LQG is optimal for White Gaussian Noise, whereas noise needs to be coloured for AI, in order to make derivatives. Therefore, a comparison is provided on two bases: the setting of LQG control; and the setting of AI. The optimal LQG controller is obtained for both settings, and AI is applied to both settings as well. When AI is reduced to the setting of LQG, an equivalent expression can be obtained by a proper choice of tuning parameters. This entails choosing a matrix such that the closed-loop is stable, which contrasts LQG control, which is always a stabilizing controller. When LQG is extended closer to the normal AI setting, we find that tuning of AI becomes harder for more complex systems, but that AI is mostly able to show optimal behaviour.

This MSc thesis presents the development of a viewpoint optimization framework to face the problem of detecting occluded fruits in autonomous harvesting. A Deep Reinforcement Learning (DRL) algorithm is developed in order to train a robotic manipulator to navigate to occlusion-free viewpoints of the tomato-target. Two Convolutional Neural Networks (CNN), You Only Look Once (YOLO) version 3 and Mask Regional Convolutional Neural Network (Mask R-CNN), are trained and evaluated in order to obtain visual information from the tomatoes. The two trained CNN achieve high detection accuracy and surpass other fruit detection methods. The instance segmentation Mask R-CNN result and an image processing algorithm are applied in an occlusion modelling method. The vision-based reward formulation for the DRL algorithm is closely related to the occlusion modelling metric. The DRL algorithm is trained and evaluated in a simulation environment by mounting a camera on a robotic arm. The use of different reward scheme and exploration strategy during DRL training shows their effect on training performance. The DRL training performance is assessed by illustrating the maximum visible tomato-target fraction per episode, the steps needed per episode, the trajectory of the robot's end-effector, and the initial and final tomato-target viewpoint in each episode. The evaluation results show a satisfactory performance, which depends on the reward and exploration strategy, as in the majority of the cases the robot can fully see the tomato-target after a few steps.

This thesis is motivated by evolutionary robot systems where robot bodies and brains evolve simultaneously. In such a robot system, `birth' must be followed by `infant learning' by a learning method that works for various morphologies evolution may produce. Here we address the task of directed locomotion in modular robots with controllers based on Central Pattern Generators. We present a bio-inspired adaptive feedback mechanism that uses a forward model and an inverse model that can be learned on-the-fly. We compare two versions (a simple and a sophisticated one) of this concept to a traditional (open-loop) controller using Bayesian Optimization as a learning algorithm. The experimental results show that the sophisticated version outperforms the simple one and the traditional controller. It leads to improvement in performance and more robust controllers that cope better with noise.

Accessibility of offshore structures is strongly affected by local weather and wave conditions. During rough wave climates, vessel motions prevent people to be transferred safely from and to an offshore structure. To increase accessibility, Ampelmann developed a motion compensating gangway system. The system uses a 6 DoF motion sensor to measure the vessel motions in order to compensate for them. This motion sensor has a sufficient accuracy to compensate the Ampelmann, but it has some drawbacks. The motion sensor is expensive - it is the most expensive part of the whole Ampelmann - and the sensor is on the blacklists of many boarder securities since the accuracy is high enough to guide a missile. Due to these drawbacks, it is interesting for Ampelmann to investigate alternative sensors. During onshore tests the currently used motion sensor is compared with an alternative. It was found that the rotations were measured accurately by both sensors, but that the transla- tion measurement was significantly worse with the alternative motion sensor. Through these findings, the focus of this research is on estimating the translations. Also, in literature was found that using multiple sensors improves the performance of the measurement. This research investigates using multiple inertial measurement sensors for estimating the translations. Noisy acceleration measurements are simulated to replicate real IMU measurements up to five IMUs. A mass-spring-damper model is used to generate a excited sine, which represents a wave. Through a Kalman filter with a Wiener model the translations are estimated from the accelerations. A pullback method is introduced, where a virtual zero measurement is added to compensate for the drift introduced when double integrating accelerations. Five different fusion techniques are investigated and compared. Through the VAF and the average error the performances of the methods are compared. The direct fusion method performed the worst. The best results for the average error were found when using the post-prediction fusion method with 3 IMUs. The best results for the VAF were found using the post-correction fusion method with 2 IMUs. Also real ship motions that were measurement offshore were transformed into one acceleration measurement, which were used in the direct fusion method and the averaging fusion method (since there is only one measurement available). Both methods gave the exact same results. The different fusion techniques are also applied on real IMU measurements on a test setup. The test setup is a translational sled on a rails. One IMU is used to get real data measure- ments. Rotating this IMU with repect to the translational movement of the sled allows us to get two measurements of the IMU. Again the direct fusion method performed the worst. The best results were found when using the post-prediction fusion method with 3 IMUs. When comparing the simulations and the real tests with the several fusion methods, the same trend is found in both the VAF and the average error. When using the post-prediciton fusion method or the post-correction fusion method an almost similar VAF is found in both simulation and test. Though the average error for each method was significantly smaller in simulation than found in tests. Based on the outcome of the simulations and the tests it can be concluded that two fusion methods are found to give good results for estimating translations from low-cost IMUs. These methods use a Kalman filter with a Wiener model. Also, a pullback method is used to compensate for drift. Before implementing this motion sensor into the Ampelmann system, further research should be done on the performance of the methods on real ship motions.

Current robots consume a lot of energy. The work required for a task is generally just a tiny fraction of the total energy consumed. Most energy wastefully dissipated. Design of the 2 degree of freedom (DoF) Plugless Robot Arm shows that clever design can reduce these energy losses to a tiny remaining fraction. The Plugless Arm is designed to perform a pick-and-place task displacing packages of 1 kg over a vertical distance of 1 m and some additional horizontal distance in a way that is capable of powering the full system. Maximum energy recovery is achieved by the design of springs in parallel to the actuators such that minimal actuator currents lead to minimal electric heat losses. This requires simultaneous optimisation of the trajectory and the parallel elastic elements. In a Cartesian configuration a novel method is tested which allows for implicit optimisation of an elastic element without additional parameterisation. The optimal parallel spring characteristic can be expressed as a pure function of the trajectory if the system dynamics are defined as a function of position instead of time. Afterwards the analytical optimal spring is replaced by a mechanism of low mechanical complexity showing similar energy characteristics. The mechanism is translated to a polar equivalent for the 2 DoF arm and optimised together with the trajectory of the arm. The optimal nominal trajectory is followed under undisturbed conditions when applying a pre-computed feedforward control signal to the actuators of the arm. Additional local optimal linear state feedback control is computed by means of Differential Dynamic Programming. All optimisation is done offline. Performance of the controller under disturbed conditions is tested in terms of accuracy as well as energy consumption. The system achieves a nominal energy retrieval of 3.64 J per cycle of 2.5 s, which is sufficient to power a light controller, the necessary sensors and a special energy efficient gripper. Some energy is left which can be used by the controller to recover from disturbances which extract energy from the system. The controller is also able to recover additional energy from disturbances which add energy to the system. The implementation of the controller is further optimised to achieve minimal computational energy consumption and memory requirements. By selective reduction of the control law, considerable data reduction is achieved with negligible impact on the controller performance.

This work addresses the problem of exploration and coverage using visual inputs. Exploration and coverage is a fundamental problem in mobile robotics, the goal of which is to explore an unknown environment in order to gain vital information. Some of the diverse scenarios and applications in which exploratory robots can have a significant impact include search and rescue missions, environmental monitoring, and space exploration. Specifically, we focus on the aspect of finding areas of interest, also referred to as targets, in the environment. In this thesis, we propose a deep reinforcement learning based approach relying solely on visual observations. In particular, our method builds upon the off-policy, actor-critic, Importance Weighted Actor-Learner Architectures (IMPALA) framework by including a set of novel auxiliary tasks, i.e. Pose Estimation and Local Map Prediction. These auxiliary tasks are inspired by Simultaneous Localization and Mapping (SLAM) approaches to exploratory robotics problems. The intuition is to assist internal representation learning and build locale specific knowledge by teaching the agent to predict its position and orientation, as well as transfer the visual information to information about the its local proximity. Experiments conducted in the DeepMind Lab simulation environment show improved performance over the base IMPALA agent and demonstrate the effectiveness of these auxiliary tasks. Furthermore, we investigate the performance of the agent, trained through various stages of curriculum, compared to a human controlled agent. The trained agent is shown to outperform the human in the majority of tested scenarios.

For many systems, like medical devices, nuclear reactors, and transportation systems, an adequate maintenance optimization approach is essential to ensure high levels of reliability and safety while keeping operational costs low. A promising approach towards this goal is condition-based maintenance, which plans maintenance only when the system health indicates a need for it. To infer the system health, monitoring devices are installed to collect health-related data. The path from the monitoring data to a maintenance schedule then involves the following steps: 1. fault diagnosis, i.e. detecting abnormal system behavior and identifying its cause; 2. failure prognosis, i.e. predicting future system health; 3. maintenance optimization, i.e. determining the required type of maintenance as well as the optimal time to perform the maintenance task. Although various methods have been published for all three tasks, discrepancies still exist between the assumptions made in the literature and the conditions encountered in practice. These discrepancies include, e.g., unrealistic assumptions regarding the absence of component interdependencies and regarding the (number of) available monitoring signals. This thesis contributes to resolving these discrepancies by proposing methods for fault diagnosis, failure prognosis, and maintenance optimization, particularly focusing on narrowing the gap between theory and practice. When treating the individual tasks, the dependencies between fault diagnosis, failure prognosis, and maintenance optimization are explicitly taken into account. @en

Control design for modern safety-critical cyber-physical systems still requires significant expert-knowledge, since for general hybrid systems with temporal logic specifications there are no constructive methods. Nevertheless, in recent years multiple approaches have been proposed to automatically synthesize correct-by-construction controllers. However, typically these methods either result in enormous look-up tables, require online optimization, or are highly dependent on expert-knowledge. The goal of this thesis is to propose a novel approach that overcomes these limitations, i.e. to propose a framework for automatic controller synthesis, capable of synthesizing closed-form controllers for hybrid systems with temporal logic specifications, without a heavy reliance on expert-knowledge. To this end, we draw inspiration from the human design process and utilize two methods that show great similarities to it, namely evolutionary algorithms and counterexample-guided inductive synthesis (CEGIS). Specifically, we use genetic programming (GP), an evolutionary algorithm capable of evolving entire programs. This makes it possible to automatically discover the structure of a solution. Moreover, it enables the synthesis of compact closed-form controllers, circumventing the need for look-up tables or online optimization. In combination with GP, we use the concept of CEGIS to refine candidate solutions based on counterexamples, until the controller is guaranteed to satisfy the desired specification. In this thesis we propose two CEGIS-based synthesis frameworks, which differ in the employed verification paradigms, namely utilizing either (co-synthesized) Lyapunov-like functions or reachability analysis. Both frameworks result in correct-by-construction compact closed-form controllers, where the use of expert-knowledge is optional. Both frameworks are capable of synthesizing sampled-data controllers, enabling implementation in embedded hardware with limited memory and computation power, forming a stepping stone towards faster automation.@en