The human brain, with its intricate web of billions of neurons and trillions of synaptic connections, is a remarkable organ responsible for performing complex cognitive processes. While brain imaging techniques like fMRI and EEG provide insights into neural activity, there is no broadly accepted mathematical framework for the collaborative activity of neuronal populations and their communication. In the interdisciplinary fields of neuroscience and computational modelling, this research introduces a data-driven mathematical framework for modelling the brain's cortical network, derived from EEG data, while incorporating an exogenous input. This framework captures brain dynamics, forming a state-space model using subspace identification, and evaluates statistical dependencies among brain sources, known as functional connectivity. Employed on a dataset mimicking a sensorimotor task, it proves effective in characterizing brain dynamics, with the PO-MOESP method outperforming N4SID. Additionally, the framework is competent to assess functional connectivity between brain sources, resulting in a network connectivity diagram that offers valuable insights into the statistical relationships between distinct brain regions. Altogether, it is concluded the designed algorithm can determine the intricate interactions within the brain during a simple passively performed task stimulating the brain. Consequently, this achievement forms a robust foundation for advancing Brain-Computer Interfaces (BCIs) and enhancing the diagnosis of neurological disorders.

Many recent robot learning problems, real and simulated, were addressed using deep reinforcement learning. The developed policies can deal with high-dimensional, continuous state and action spaces, and can also incorporate machine-generated or human demonstration data. A great number of them depend on state-action value estimates, especially the ones in the actor-critic framework. Deriving unbiased estimates for these values is still an open research question, mostly since the connection between accurate value estimates and system performance is not yet well-understood. This thesis work has three main research contributions. Firstly, it analyzes the connection between value estimates and performance for the TD3 algorithm. Secondly, it derives theoretical bounds for the true value function when dealing with environments where a reward is only given for successful completion of a task (sparse/binary reward). Lastly, a deliberate underestimation objective is added to the TD3 algorithm together with the theoretical bounds to improve system performance when using human demonstration data that only covers a specific part of the state and action space. All the algorithms are tested and evaluated using simulated robot manipulation tasks in the robosuite environment, where the robot is first trained on the demonstration data and then can gather more experiences in the simulation. Results show that the deliberate underestimation together with the value bounds enable the robot to learn from human demonstration, which was not possible for the standard TD3. Additionally, applying just the value bounds speeds up the learning process when using machine-generated datasets.

To meet the demanding requirements on the environmental impact of aircraft, radically new aircraft concepts need to be developed. Within the NOVAIR project, Royal Netherlands Aerospace Centre (NLR) tests these new concepts on a Scaled flight demonstrator (SFD). Using an SFD allows for testing of dynamic and flight physical behavior of new aircraft concepts, which is difficult with more theoretical methods. Furthermore, by using an SFD, the risks associated with full-scale testing in terms of cost and time are minimized. One of the new concepts developed for the SFD is Distributed electric propulsion (DEP). Here, the two jet engines are replaced by six electric propellers. As these can be used actively for control, this results in an over-actuated system. These propellers interact with the aerodynamics of the wing resulting in Propulsion airframe interaction (PAI) effects. Although using the PAI effects for control has the potential to improve capabilities, efficiency and robustness of the aircraft, research into controllers using these effects actively is limited. This thesis, therefore, presents a new control method including control allocation for the DEP-SFD aircraft, based on the nonlinear Incremental nonlinear dynamic inversion (INDI) controller. INDI enables controlling the nonlinear dynamics of the DEP aircraft over the complete flight envelope with one controller. By feeding back real-time sensor measurements, robustness to modeling errors and external disturbances is increased. Using the Incremental nonlinear control allocation (INCA) method, the full control authority of the DEP can be used. This technique enables taking into account nonlinear allocation relations and control effector interactions, while solving the control allocation real-time, which is key in actively using the PAI effects for control. The INCA method is used for two performance improvements: tracking performance and propeller power efficiency. To compensate for actuator dynamics, this thesis implements an Model predictive control (MPC) controller which results in improved tracking and higher efficiency. The performance of the controller was analyzed in simulation, where a reference square signal input on the roll angle was applied, while minimizing the sideslip angle and maintaining a steady altitude and velocity. The INCA controller with MPC is compared to a conventional INDI controller, showing a significant decrease in rise time from 2.46 s to 0.703 s with minimal tracking error. Furthermore, the effective bandwidth of the system was increased from 0.186 Hz to 0.663 Hz and the power consumption reduced by 6.3%. Modeling uncertainties, external disturbances and a propeller fault were introduced to verify the robustness of the controller. Finally, the reference altitude and velocity were varied, demonstrating controller performance over a large part of the flight envelope.

Vehicles are becoming increasingly complex and the resource-demanding due to the developments aimed at driving autonomy. Viewing an automobile as a cyber-physical system (CPS), this work applies the advances of controls applied to CPS for improving resource-consumption. One such idea is that of event-based control. Theoretical developments in event-based control show promising results, but they require practical corroboration. Combining these aspects, an event-triggered vehicle stability controller is designed and tested on a real-time hardware-in-loop setup.

This research aims at quantifying the uncertainty in the predictions of tensor network constrained kernel machines, focusing on the Canonical Polyadic Decomposition (CPD) constrained kernel machine. Constraining the parameters in the kernel machine optimization problem to be a CPD results in a linear computational complexity in the number of features, whereas the original problem suffers heavily from the curse of dimensionality as the number of parameters scale exponentially. By employing a product feature map with polynomial features, the original data input is transformed to a higher-dimensional space. Three different methods are investigated for quantifying the uncertainty of the predictions of the CPD constrained kernel machine. Firstly, the delta method is proposed which is a frequentist approach that linearizes a nonlinear parametric model around the estimated model. By estimating the covariance of the model parameters, the delta method can estimate the uncertainty in the model predictions based on the estimated parameter uncertainties. The delta method is compared to two other methods that are able to reflect the prediction uncertainty: the Bayesian method and Single Bayesian Core (SBC) method. The Bayesian method treats the parameters in the factor matrices of the CPD as probability distributions rather than single values and the SBC method incorporates both frequentist and Bayesian aspects. A comparison between the three different methods is performed based on an assessment on the correctness and informativeness of the uncertainty measures of prediction intervals and confidence Intervals. It was found by regression and classification experiments that all three methods can provide valuable uncertainty quantification measures in terms of correctness and informativeness for the CPD constrained kernel machine. However, the Bayesian method provides in general more conservative uncertainty intervals compared to the delta and SBC method. A major drawback of the Bayesian method is its lack of scalability as the size of the mean and covariance, constructed by the unscented transform in the Bayesian method, scale exponentially. Furthermore, the delta and SBC method produce high quality uncertainty intervals and the methods provide remarkably similar uncertainty quantification on the prediction error variance.

This research analyses the potential of Electrical Signature Analysis (ESA) as a means for condition monitoring of mechanical defects of induction motor driven thruster assemblies in the maritime industry. This is done by measuring the three phase current and voltage of three thrusters, with different health conditions, of a marine vessel sailing at different speeds. In this data, the mechanical behaviour of the machines is detected by plotting the frequency spectra of various electrical parameters, such as complex current and torque. After successfully identifying mechanical behaviour and defects, a suggested approach for the implementation of ESA as a means for condition monitoring is presented.

There is growing interest to control cyber-physical systems under complex specifications while retaining formal performance guarantees. In this thesis we present a framework for formal control of uncertain systems under complex specifications. We consider dynamical systems with random disturbances, whose probability distribution is unknown. When it comes to the specifications, we focus on those given as syntactically co-safe linear temporal logic (scLTL) formulas. Such formulas resemble natural language, and allow us to reason over complex behaviours of the system. We follow an abstraction-based approach: we abstract the original system to a finite-state Markov model, in which the state discretization error as well as the distributional ambiguity, are embedded as uncertainties. To do so we make use of tools from optimal transport and ambiguity sets of probability distributions. After that, we synthesize a strategy for the abstraction and obtain probabilistic guarantees that the abstraction satisfies the specifications. Finally, we correctly refine the strategy to one that the original system can follow, and prove that the guarantees we obtained for the abstraction also hold for the original system. We propose two approaches to obtain the abstraction. First, we propose a data-driven approach to abstract the system into an interval Markov decision process (IMDP) when samples from the unknown distribution is available. Then we use already existing algorithms to synthesize a strategy for the IMDP. Secondly, we propose an approach to abstract the original system into a robust Markov decision process (robust MDP). This second approach is applicable to more general uncertainty models besides the data-driven one, and reduces conservatism of the abstraction. Furthermore, we propose an algorithm to synthesize robust strategies for robust MDPs, which also renders the guarantees that the abstraction satisfies the specifications. Finally, we demonstrate the usefulness of our proposed approaches through several case studies that involve both linear and nonlinear systems.

The high-tech industry continuously pushes the boundaries of controller performance to achieve faster and more precise machines. Currently, linear control is the standard in the industry. These controllers suffer from the waterbed effect and Bode's phase/gain relation, which impose inherent limitations on the precision and robustness of the system. Reset control is a popular strategy to get around these limitations and improve performance. The damping in reset control systems is not only determined by the phase margin of the system but is also dependent on the exact controller element sequence. Currently, finding the optimal controller sequence is done through simulation of the step response. However, this fails to provide insight into the underlying cause of the additional damping achieved by specific controller configurations. This thesis proposes an analytical approach to analyze the damping of the transient response reset control systems. The analytical analysis provides a better understanding of sequence-dependent damping and assists in controller design. First, the analytical expressions of the step response and states of the system are derived, which are used to define the system's energy. The step response and energy equations are used to characterize the damping in a reset control system. To show the value of the proposed method, the damping in a reset control system is assessed as an illustrative example. It is found that when a lead is in front of a reset element, the reset controller can provide more damping because it reduces the oscillatory content in the step response.

To extract energy from wind, both Horizontal Axis Wind Turbine (HAWT)s and Vertical Axis Wind Turbine (VAWT)s are used. Different from the HAWT, the axis of rotation of the VAWT is perpendicular to the ground. This vertical design offers some unique advantages for the VAWT. For example, the generator of the VAWT is located on the ground, hence the maintenance costs are lower compared to the HAWT. Furthermore, the VAWT can be upscaled easier than the HAWT to harvest more energy from a single turbine. These benefits demonstrate the potential of VAWTs for offshore applications. However, the VAWT’s blade suffers from periodic aerodynamic loads from the periodically varying Angle of Attack (AOA), and unsteady aerodynamic phenomena, which harm the durability and hence the economic feasibility of the VAWT. This thesis aims at resolving the blade loads mitigation problem of the VAWT. For wind turbines, pitching the blade is a common way to reduce the loads on the blade. There have been studies done that focus on the power maximization for the VAWT. However, research on the pitch control system for load reduction is lacking. To tackle the blade loads mitigation problem, a data-driven approach called Subspace Predictive Control (SPC) is used to control the individual pitch actuation of a two-bladed 1.5 m H-Darrieus VAWT. This closed-loop control strategy consists of two parts, an online identification block and an optimal control law based on the identified system. The online identification method applied is called Recursive Predictor-Based Subspace Identification (RPBSID). It can estimate the system parameters online by using the input and output data collected from the VAWT system. Then, two different types of controllers are designed to reduce the blade loads. In the first approach, the constrained Model Predictive Control (MPC) is used. The optimal control input is calculated by solving a finite horizon constrained optimization problem. The individual pitch control action can be automatically adapted based on the online identified model. In the second approach, a Linear-Quadratic Regulator (LQR) is used as a comparison to the first approach. The LQR solves an infinite horizon unconstrained optimization problem to obtain the optimal state-feedback law. The control gain can be calculated analytically. Both approaches are first tested on a Simulink model, which is based on the interpolation of lookup tables computed by the Actuator Cylinder (AC) model and blade element theory. Under a stepwise wind condition, the SPC approach shows the potential to reduce the normal load on each blade. It is also found that the MPC is more robust and easier to implement, compared to the LQR. Then, the MPC based SPC is applied to a mid-fidelity simulation tool called Qblade to acquire more realistic results. The controlled pitch trajectory shows good load reduction results under both uniform and turbulent wind conditions. The work presented in this thesis provides a control strategy to reduce the blade loads on VAWTs following a data-driven manner and demonstrates the capability of MPC based SPC in VAWTs’ application.

The scheduling algorithm of the printer is an important factor that affects printing efficiency. For current printers, paper scheduling often follows the first-in-first-out principle, so it is often not optimal. The printer system is a type of semi-cyclic discrete-event system with synchronization but no concurrency. The system contains a set of operations that vary over a limited sequence of operations, which can be further modelled as a Switching Max-Plus-Linear (SMPL) system through max-plus-linear algebra, and can be switched between different modes of operation. Since max-plus algebra has a significant analogy with conventional algebra, some properties in conventional algebra can be applied to such kind of system, making the system easier to achieve optimal control. In this report, a printing scheduler modelled by the max-plus algebra is presented. We review and summarize the basic knowledge of the Max-Plus-Linear (MPL) system from the existing literature in the first half of the report. We first introduce the basic properties of max-plus algebra and SMPL systems. Then we turn to the stochastic case, and consider the two types of stochastic uncertainty that may be included in the SMPL system, namely stochastic parametric uncertainty and stochastic mode switching uncertainty. Next, we review a Model Predictive Control (MPC) approach that can achieve optimal scheduling of systems containing two random uncertainties. In the second half of the report, based on the content summarized in the first half, we make a further derivation of the printer scheduling system modelled by SMPL approach. We first present the modelling framework of the printer. Three working modes, namely duplex mode, idle mode and simplex mode, are modelled separately and merged into a compact form. Then a scheduler that takes the feeding and processing time of each sheet of paper as design variables is introduced. It can find the global optimal schedule of different types of paper by solving the Mixed-Integer Linear Programming (MILP) problem. After that, we discuss cases involving switching, and consider switching between two sizes of paper and two working modes. Three possible intermediate switching modes are designed. Finally, we take noise/interference into consideration, discuss the impact of noise on scheduling, and the changes that the previously proposed methods may require to achieve optimal scheduling in the presence of noise.

Event-Triggered Control (ETC) is a control method where the controller is only updated when necessary. The control inputs are kept fixed until a state-dependent event triggers their re-computation. The triggering condition is designed to guarantee the stability and desired performance of the control system. This prevents the excessive use of scarce communication and energy resources. To fully take advantage of these savings when dealing with a (physically) distributed system, it is vital to decentralise the triggering condition. A decentralised triggering condition can be checked locally at the nodes of the system using the locally available states, eliminating the need to send updated sensor measurements to a central location constantly. However, the decentralisation of the triggering condition needs to be optimised for the state of the system to fully take advantage of the possible resource savings. This thesis provides a framework to optimally decentralise quadratic triggering conditions for Periodic Event-Triggered Control (PETC) systems in the absence and presence of bounded disturbances. In addition, a method of constructing a region-based map of such optimisations is provided, making it feasible to implement the optimisation approach on larger systems without needing more powerful hardware.

Wind energy becomes more and more popular since it is environmentally friendly. Wind farm control is one of the most popular topics and it works on steering the wind farm to extract energy from wind as much as possible. Generally, the model capturing wake effects between turbines in the wind farm plays a role in wind farm control. The existing FLORIS model is considered suitable for wind farm control due to the fact that it has the ability or potential to capture wake features with reasonably computational costs. A drawback of the FLORIS model is the lack of dynamics, which is improved by developing the FLORIDyn model. This thesis focuses on a Gaussian FLORIDyn model. The objective is to explore the possibility of improving the model accuracy by quantifying the associated uncertainty in the model parameters. Uncertainty quantification consisting of sensitivity analysis and Bayesian calibration is conducted based on a 3-Turbine case simulation using the UQLab software. Since a MCMC algorithm associated with Bayesian calibration requires to evaluate the FLORIDyn model multiple times, it can result in massive computational expenses when directly applying the computational model to the simulation. To deal with this, a surrogate model is first constructed to replace the original model. This thesis assesses two types of approaches for surrogate model construction which are the Kriging-based approach and the PCE-based approach. One approach is chosen after the comprehensive comparison in terms of accuracy and efficiency. The constructed surrogate model is then applied to the sensitivity analysis using Sobol' indices to investigate how each model parameter of interest affects the model output. Last, the high-fidelity SOWFA data are used as experimental data for Bayesian calibration. Compared to non-calibrated model outputs, calibrated model outputs are closer to the SOWFA data, which means that the accuracy of the FLORIDyn model is improved.

This thesis investigates the potential of state-dependent sampling strategies (SDSS) for the control of heavy-haul trains. Event-triggered control (ETC) is a control approach in which data is only sent when some state-dependent condition, the triggering condition, is satisfied. In this way, the number of communications required to stabilise a system can be drastically reduced. Periodic event-triggered control (PETC) is a variant of ETC in which the triggering condition is checked periodically. By sometimes sampling earlier than a PETC-generated deadline, long-term pay-offs in the average inter-sample times are possible. As searching for formal models of early-triggering controllers quickly becomes computationally infeasible as system dimensions grow, a sample-based approach using reinforcement learning was utilised to find the SDSS controller, which takes the form of a neural network mapping states to the time until the following sample, trained to hopefully yield long-term payoffs in sample efficiency. It was found that, by choosing suitable control parameters, the SDSS controller can outperform the PETC baseline in terms of inter-sample times (IST). Next, a hardware-in-the-loop (HIL) setup was made to evaluate the optimised controller’s performance in a real-time context controlling a non-linear train system subject to noise, disturbances, and tasked with attaining different speed setpoints while minimising the inter-wagon forces. It was found that the optimised controller did not perform well during these scenarios. Since the controller’s robustness was not considered during the training process, performance suffered under these conditions. To improve the robustness of the controller, the system must be trained on realistic (noisy) data. Simulations with more wagons must be performed to assess whether improvements can still be attained when using larger models. Finally, the accuracy of the HIL setup needs to be improved by using an appropriate network stack that would allow each wagon to send its own data and turn off the sensor node radios when otherwise idly listening for incoming data.

In multi-agent systems reaching consensus has been a long-standing problem. A considerable amount of research has been focused on how event-triggered consensus can be used to limit the energy consumption of the communication system while still ensuring convergence to the neighbourhood of a point or a formation. These algorithms rely on having knowledge of the global state information of each agent in the network. Given this information, it is possible to consider using distributed algorithms from the field of Computer Science to achieve the same. This thesis presents a study on average consensus algorithms for single-integrator multi-agent systems. The focus is on comparing the time required to reach convergence and the energy required by the communication system. To perform this study, three methods are used to compare the algorithms. The first method involves deriving mathematical bounds on the convergence time and the number of transmissions. The second involves a simple simulation to verify the mathematical bounds. Finally, the algorithms are compared in a practical application where robots are subjected to noisy measurements and need to assemble in a formation. The results of the comparisons show that distributed algorithms are capable of fast convergence and require less energy for communication. Therefore, it is worth considering the use of distributed algorithms when agents have enough memory to temporarily store the global state of all agents in the network. There might still be situations in which the ETC algorithm converges slightly faster but this benefit is overshadowed by the increase in energy consumption.

Robot dexterous manipulation research has drawn more attention in recent years since the development of various learning methods makes it possible for robots to achieve dexterity at the human level. Many attempts have been made to integrate human knowledge into Reinforcement Learning (RL) processes for faster learning speed and better performance. Despite their successes in many aspects, there are two open problems that still need to be carefully considered: 1. The effect of demonstrations gradually vanishes during RL. 2. In most cases, only imperfect demonstrations are available to robots. In this work, we proposed a new learning framework - Interactive Behavioural Cloning for faster Reinforcement Learning (IBC-RL), which could alleviate problems in complex manipulation tasks with long horizons. Different demonstrations are shown to robots at different learning stages. Robots learn complex tasks step by step with interactive demonstrations from human teachers. The framework is evaluated with four dexterous manipulation tasks simulated with the Isaac Gym engine. Human teachers perform demonstrations by controlling the simulated robot hands through a hand-tracking system. The results of the experiments could demonstrate the efficiency of IBC-RL in guiding and accelerating the learning processes with imperfect demonstrations.

Undesired vibrations are one of the most significant sources of error in any type of mechatronic system or component. The emerging field of elastic (locally resonant) metamaterials offers a viable solution to successfully suppress these by generating bandgaps in both resonant and non-resonant regions. In this thesis, metamaterials in a sensor-actuator configuration using piezoelectric transducers are employed to generate vibration attenuation regions in beam-like structures. The main contribution of this thesis consists in studying the practical issues involved in the experimental implementation of such metamaterial architectures, often overlooked in the literature. To this aim, parasitic dynamics such as time delay and RC roll-off characteristics of piezoelectric transducers are considered, and their influence on controller choice is evaluated. The research was conducted using full model simulations in SPACAR and an experimental setup. The RC roll-off characteristic of piezoelectric transducers was found to be significant in limiting the bandgap generation capabilities of the system in non-resonant regions. The reason for this was the added phase caused by the parasitic effect, which required a reduction in controller gain for stability and ultimately reduced the bandgap performance. This was not the case for resonant bandgaps, where the phase lead was compensated by the increase in gain at the resonance. This ultimately allowed for optimal resonant bandgaps to be generated and observed. Methods to compensate for such parasitic effects are proposed and suggestions on how to implement these to attain non-resonant bandgaps are made.

Noise's impact on biochemical systems has long been a focal point of investigation, given its potential to compromise signal accuracy and disrupt system functionality. This paper conducts a comprehensive exploration into the noise characteristics within a set of signal differentiators recognized for their high precision. Noteworthy for their modularity, swift computation, and ease of implementation, these differentiators play a pivotal role in computing concentration changes and bear the potential to regulate the dynamics of biological systems. This study establishes a comprehensive simulation framework to examine the noise characteristics of these differentiators across diverse input signal scenarios. Furthermore, we also apply noise suppression techniques such as noise filters to mitigate excessive noise and enhance noise performance. Our findings reveal that these differentiators significantly amplify the system noise level, surpassing both the Poisson level and the original system noise level. Moreover, while noise filters demonstrate notable success in noise reduction, achieving Poisson-level noise without compromising signal integrity remains a challenge. This investigation yields invaluable insights into the noise properties of biochemical differentiators, shedding light on their inherent limitations. Additionally, it presents a viable pathway to enhance noise behaviour, thereby extending the scope of applications for these differentiators.

The increased diffusion of Renewable Energy Sources (RES) into energy distribution systems gives rise to a number of issues that the Distribution Network Operators (DNOs) need to face. The dependency of RES on weather, along with their intermittent and non-dispatchable nature, urges us to develop frameworks that can guarantee a stable network, despite the power fluctuations. In this thesis, we develop a Data-Driven Optimal Power Flow (OPF) formulation in order to achieve voltage regulation against operational problems that may occur because of the high PV penetration into energy distribution systems. The unpredictable PV generation deteriorates the system’s reliability and introduces instability. Thus, we develop appropriate formulations to define a limit on exports from residential PV owners to the grid. For this purpose, we employ Distribunally Robust Chance Constraint Programming (DRCCP), as a method that can handle constraints that depend on the uncertain PV generation and residential demand. We capture the distributional uncertainties with an ambiguity set and we utilize the Wasserstein metric to parameterize the range of this set. We divide the thesis into two case studies. In the first case study, we perform the DRCCP optimization with only few recorded data. We evaluate the results of our algorithm in a dataset of 500 days of recorded data and we achieve to reduce significantly the overvoltage instances. We tune the DRCCP in a way that we achieve at least 95% satisfaction of constraints that emerge from the DRCCP, avoiding overconservative solutions and high generation cuts that could harm the PV owners. In the second case study, we intend to exploit plenty of historical data, but the computational burden hinders us from using them in the DRCCP. Therefore, we employ Wasserstein Barycenters, and we utilize the Wasserstein distance in order to cluster the data. With Wasserstein Barycenters we reduce significantly the running time and we provide an efficient and robust output, for at least 95% satisfaction of the model’s constraints.

With the use of simulation models, predicting and optimising the correct dynamic behaviour and parameters of a propulsion system of a ship can be performed cheap and safe. However, capturing the right dynamic behaviour is very difficult. Besides, building simulation models and determining the correct parameters is a time consuming process. With system identification, in- and output data of a controlled test are used to identify the parameters of the created grey box model structure, which reflects the underlying physical laws. A so called "fingerprint" is generated that imitates the behaviour of the system. The first attempt of system identification of a full-scale propulsion system showed promising results but asked for further research. This thesis further investigates if system identification is a suitable method to obtain the dynamic model- behaviour and parameters of a full-scale propulsion system in a short time, with the use of controlled tests.