The rapid integration of renewable energy sources, such as wind and solar power, into modern electricity systems has introduced challenges in balancing supply and demand, managing grid congestion, and ensuring efficient energy market participation. This thesis develops a framework for optimizing the portfolio management of hybrid power plants (HPPs) under uncertainty. HPPs combine renewable generation, energy storage (batteries), and flexible demand (electrolyzers) to improve grid efficiency and market participation.

This thesis develops a framework for optimizing HPP portfolio management under uncertainty, specifically targeting participation in the day-ahead market, the Dutch aFRR market, and the strategic use of passive imbalance. Two complementary optimization methods were developed and compared. Stochastic programming (SP) manages uncertainties in wind generation, activations, and day-ahead market prices using scenario-based approaches. Adaptive robust optimization (ARO) employs worst-case scenarios defined by budgeted polyhedral uncertainty sets for wind generation and activations, while using scenarios for day-ahead market price uncertainty. The ARO framework leverages duality theory and a column-and-constraint generation algorithm (CCGA) to iteratively refine robust solutions. Both methods were implemented within a shrinking horizon framework, which allowed for iterative decision-making at each time-step and tested the robustness of the first-stage decisions.

A case study using real-world Dutch market data demonstrated that both methods achieved zero violations by leveraging passive imbalance. ARO consistently delivered more robust first-stage decisions and higher revenue by effectively utilizing activation capacity. Additionally, a parameter study revealed the trade-offs between robustness and revenue, offering valuable insights into the flexibility and effectiveness of each method under varying conditions. This work highlights the potential of hybrid power plants to improve operational reliability and profitability in modern electricity systems

This paper introduces a new imitation learning framework based on energy-based generative models capable of generating complex, life-like, physics-dependent motions, through state-only expert motion trajectories. Our algorithm, called Noise-conditioned Energy-based Annealed Rewards (NEAR), constructs several perturbed versions of the expert's motion data distribution and learns smooth, and well-defined representations of the data distribution's energy function using denoising score matching. We propose to use these learnt energy functions as reward functions to learn imitation policies via reinforcement learning. We also present a strategy to gradually switch between the learnt energy functions, ensuring that the learnt rewards are always well-defined in the manifold of policy-generated samples, thereby improving the learnt policies. We evaluate our algorithm on complex humanoid tasks such as locomotion and martial arts and compare it with state-only adversarial imitation learning algorithms like Adversarial Motion Priors (AMP). Our framework sidesteps the optimisation challenges of conventional generative imitation learning techniques and produces results comparable to AMP in several quantitative metrics across multiple tasks. Finally, a portion of this paper also analyses the optimisation challenges of adversarial imitation learning algorithms, and discusses some previously under-explored failure modes, providing rigorous empirical results to back our argumentation. Code and videos are available at anishhdiwan.github.io/noise-conditioned-energy-based-annealed-rewards/

The Optimal Power Flow (OPF) problem, a cornerstone of power system operations, has gained increased attention since its inception by Carpentier in 1962. OPF is fundamentally an optimization challenge aimed at enhancing electric power system operations within the bounds of physical and operational constraints. Over the decades, various methodologies have been explored to address the OPF problem, adapting to evolving grid complexities and the integration of distributed energy resources. These advancements have brought to the fore issues related to system randomness, fluctuation, and the need for rapid control mechanisms. This thesis introduces a comprehensive solution incorporating an online optimization algorithm tailored for real-time OPF applications. This approach, characterized by minimal computation times, integrates a feedback strategy that obviates the necessity for instantaneous power demand information and employs a Shape-constrained Gaussian Process for the estimation of unknown cost functions. The proposed control algorithm demonstrates robust tracking performance and satisfactory computation efficiency, marking a significant improvement towards optimizing future power networks fraught with increasing size and complexity. Moreover, this work delves into the investigation of various system design parameters, offering insights into potential avenues for enhancing system performance. Through a meticulous examination of these parameters, the thesis sheds light on strategies to refine the integrated system’s efficacy, paving the way for more resilient and efficient power networks.

Active knee prostheses are potent in assisting users, providing symmetry in walking, reducing metabolic costs, and preventing long-term health problems. The heart of their complex control algorithm employs the Impedance Control (IC) Law, which controls the torque output of the device by three parameters: stiffness coefficient, equilibrium angle, and damping coefficient. Ideally, these parameters should be personalized and adaptive to address interpersonal and intrapersonal variations on level-ground walking. However, current practices achieve personalization only through basic normalization and do not address adaptiveness. This thesis aims to utilize an RL framework with a novel frequency-domain state representation to address personalization and adaptiveness simultaneously. A policy iteration algorithm from the Q-learning family was chosen as the essence of the RL framework, and bellman error (BE) was chosen as the primary evaluation metric. The study revolves around two hypotheses. First, does the RL framework is suitable for the system? Second, does the frequency-domain state representation perform better than the time-domain state representation? Within the scope of the thesis, a custom environment was created by modifying the Humanoid-v04 environment of OpenAI Gym using MuJoCo (Multi-Joint dynamics with Contact). This environment is used to train the RL framework and conduct the experiments. Results suggest that the proposed RL framework can improve the system, and the frequency-domain state representation is superior to its time-domain counterpart. The latter conclusion has an impact beyond the active knee prosthesis domain and can inspire any trajectory following tasks with periodic signals.

This research presents the design and implementation of a fault diagnosis filter for a high-fidelity simulation model of the AB383 wire bonder. Fault diagnosis, which consists of detecting, isolating, and estimating faults, enables more effective maintenance strategies and potentially mitigates costly downtime in high-precision motion and positioning systems. When a system deviates from its expected behavior, it can be an indication of the presence of a fault within the system. Faults can occur as either multiplicative faults, arising from deviations in parameters within the system, or additive faults, resulting from external fault signals that impact the system’s operation. This study uses a model-based methodology that generates residuals that are subsequently analyzed using a regression method aimed at determining the influence of each fault in the residual signal, thereby facilitating fault estimation. A residual signal represents the difference between the actual system behavior and the expected behavior, mainly serving as an indicator of potential faults within the system. A data-driven threshold design is proposed to determine the detectability of faults. The main contributions of this work include the development of a fault modeling framework for the residual generation method and the application of the fault estimation framework to a linear high-fidelity simulation model affected by both multiplicative and additive faults. The simulation results demonstrate satisfactory performance, that is, accurately detecting and estimating faults. Moreover, the proposed method effectively identifies external disturbances and high levels of noise through power spectral density analysis. The findings highlight the potential of this approach, outperforming alternative methods in terms of accuracy when it comes to fault diagnosis for wire bonder machines. The research findings contribute to the active field of fault detection and estimation for complex systems, offering valuable insights for further studies and potential practical applications.

Cyber-physical systems are vulnerable to malicious attacks, which can lead to severe consequences. Active detection methods have emerged as a promising approach for identifying such attacks. However, existing active detection methods are susceptible to malicious parameter identification attacks, where attackers exploit eavesdropped data to identify and manipulate the active detection mechanisms. In this work, we propose two methods to address the issue of malicious parameter identification: the system immersion coding method and the hybrid multiplicative watermarking method. These approaches have a primal focus on disturbing the identification of attackers and defending against malicious parameter identification. Besides, as active detection methods, both of them are capable of detecting multiple attacks.

The system immersion coding method, derived from the privacy solution in federated learning, is adapted to enhance its capability to detect malicious attacks by merging the input information and defend malicious parameter identification by leveraging its privacy-preserving properties. This method involves mapping the plant output into a higher-dimensional space and introducing carefully defined noise, which can create arbitrarily large disturbances without compromising performance. The introduced disturbance disrupts the attacker's parameter estimation. Theoretical conditions are provided to discuss the detection performance of replay attacks, control-signal-injection zero-dynamics attacks, and sensor-signal-injection zero-dynamics attacks. However, we also identify that the system immersion coding method is vulnerable to known-plaintext attacks.

Watermarking is a promising active diagnosis technique for the detection of highly sophisticated attacks. Motivated by the computational hardness problems of cryptography analysis, we propose a hybrid multiplicative watermarking scheme as an active diagnosis technique. In this scheme, watermarking parameters are periodically updated based on the dynamics of unobservable states in specifically designed piecewise affine (PWA) hybrid systems. We conduct a theoretical analysis to assess the impact of this scheme on closed-loop performance, demonstrating its stability preservation. We also provide conditions to detect replay attacks and control-signal-injection zero-dynamics attacks. Furthermore, we demonstrate that the proposed approach makes it challenging for an eavesdropper to reconstruct watermarking parameters, considering both computational complexity and systems theoretic perspectives.

This master thesis introduces Hierarchical Active Inference Control (HAIC) as a control method for nonholonomic systems. This method only requires tuning of a minimal number of hyperparameters and has a relative low computation load. HAIC is based on recent research done in the application of the neuroscientific theory of Active Inference for robot control. The hierarchical aspect of this method introduces two layers of Active Inference Control (AIC) and the introduction of an additional decision hyperparameter. The top layer AIC is designed such that it takes the nonholonomic constraint into account. This causes HAIC to be able to control a nonholonomic mobile robot to a three-dimensional reference state in comparison to regular AIC which is unable to. The selection of the introduced hyperparameter gives a trade-off between robustness against noise present in the measurements and possible error in
the y-dimension of the local reference frame.

Convergence of a two-wheeled differential-drive mobile robot to a reference state using HAIC is demonstrated both by simulation and physical experiment. Simulations are done for a large range of different starting states. These simulations show the ability of HAIC to converge the mobile robot for any starting state. It also confirms that a higher value for the decision parameter causes a larger error. Another group of simulations is done investigating the new
hyperparameter introduced with HAIC. These simulations check the ability to converge the system when different noise levels are present in the measurements. It is concluded that tuning of the newly introduced hyperparameter needs to accommodate the noise present in
the measurements. Where the higher the noise, the higher the value of the hyperparameter is needed for convergence.

A physical experiment is done where the robot is controlled to five different reference states. Data obtained from this experiment shows the ability of HAIC to control a physical nonholonomic system. Like the simulations, it is also shown that a correct value is required to set the newly introduced hyperparameter. Additionally, the physical experiment shows that HAIC is not robust when disturbances can cause sudden large value changes in the measurements.

Indoor positioning systems cannot rely on conventional localization methods, such as GPS, to locate devices because of interference with the structure of buildings. One solution is to use magnetic positioning, which is based on spatial variations in the patterns of the ambient magnetic field. To model magnetic fields, Gaussian process regression is used, providing predictions of the magnetic field at unvisited locations along with uncertainty quantification. These predictions and their uncertainties are valuable information for probabilistic localization algorithms used for magnetic positioning. Full Gaussian process regression has poor scalability, becoming computationally intractable from roughly 10,000 one-dimensional measurements due to its associated cubic computational complexity. In the existing literature, approximations for Gaussian process regression have been extensively studied to reduce this computational complexity. Of these approximations, only approximations involving basis functions and local experts have been used in the context of scalable magnetic field modeling. A favorable approximation framework from existing literature uses structured kernel interpolation (SKI), allowing for fast regression through efficient Krylov subspace methods. The SKI framework is favorable as it allows for fast regression in low dimensions without introducing boundary effects. In this thesis, the SKI framework is used to approximate two distinct magnetic field models: the shared model, which considers independence between the magnetic field components with shared hyperparameters, and the scalar potential model, which includes physical properties of the magnetic field (Maxwell’s equations) in the model. The scalability of the approach is shown using simulations and experiments with magnetic field measurements. Through the simulations, it is shown the SKI framework accurately and efficiently approximates the models. The applicability of the SKI framework for scalable magnetic field modeling is investigated using data collected using a motion capture suit, the Xsens MVN Link Suit. In the final experiment, a magnetic field map is constructed based on more than 40,000 three-dimensional measurements without splitting the data set, which took less than one minute on a standard laptop.

Logistics and mobility services play a major role in our society, and efficient routing is a crucial part of this. However, even though routing problems have been widely researched, the solutions provided by algorithms do not always match drivers' expectations. Routing costs used by these algorithms are often based on one or a few parameters, but in real-world operations, many factors and sometimes hard-to-define aspects are responsible for this. Drivers can consider these different aspects and some studies found that experienced drivers often plan better delivery routes than the optimization tools. In this research, we focus on using expert decision data as examples for learning the costs of routing and train a policy that can make decisions more in line with the expectations of the expert. We formulate state-action representations for the TSP and CVRP, which we use to interpret these routing problems as inverse optimization and multiclass classification problems. Additionally, we propose multiple policy training approaches as well as state feature vector transformations that can be used based on the characteristics of the routing problems. These different training configurations are utilized to train different existing algorithms with training data sets consisting of example state-action pairs. The performance of the trained models is compared to each other and the optimal solution. The optimal solution acted as expert example and was used to create the training data. We demonstrate that both inverse optimization and multiclass classification algorithms are able to imitate expert decision-making for new problem instances from example data. However, we also show a large variation in performance depending on the problem, state features, algorithm formulations and training configuration.

The purpose of Supply Chain Management (SCM) is to maximize and stabilize the flow of goods through a supply chain. Disruptions to this flow, such as the Bullwhip Effect (BWE), have become increasingly severe and dynamic, challenging current methods for counteracting these disruptions. This thesis develops a systems and control framework for designing SCM policies to meet this challenge.

The thesis makes use of economic-engineering modeling principles to model the dynamics of the supply chain and calculate the flow of goods. Specifically, the supply chain is considered to be analogous to an electrical circuit, with the flow of goods analogous to the current and price changes analogous to voltage drops. Based on these analogies, the thesis develops the building blocks required for modeling a supply chain, consisting of storage, production and external markets. The building blocks are used to construct a serial supply chain with multiple stages and products. The supply chain is analyzed in both the time and frequency domain, quantifying how disruptions influence the flow of goods.

To regulate the flow of goods through a supply chain, this thesis models the procurement, production and product pricing policies of SCM as individual PID controllers. The PID controllers are shown to regulate the flow of goods in a manner similar to how an actual manager would.

The effectiveness of the framework is demonstrated in a simulation study of the BWE in the supply chain of Valtris Specialty Chemicals (VSC). It is shown how tools in classical control theory, like Bode diagrams, are effective in the analysis of the dynamics and severity of the BWE in terms of the resonance frequency and the amplification, respectively. In addition, this thesis shows how the tuning of the PID controllers relates to specific adjustments in the policies of VSC managers.

This thesis discusses the chemical composition (basicity) control problem of HIsarna, an experimental iron furnace which operates with 30% less CO2 emissions than its traditional blast furnace counterparts. The control challenge is keeping the basicity of the plant in a narrow operating region. A mass balance model of the plant was constructed - as it is common practice in the literature for traditional blast furnaces - which we combined with a parameter search method to find the most optimal model parameters from data. Next a stochastic system model of the plant was derived using the prediction errors of the plant on a new dataset. The novelty of our work is the chosen dynamic programming controller approach that we used for controller synthesis that enables optimal control of the plant with respect to the known model error distribution. A continuous Markov Decision Process based infinite horizon controller was devised by using Value Iteration to find a fixed point in our value function space with respect to the Bellman operator: our static value function. We used multilinear interpolation and a state and inputs grid to define our value function for our continuous state space. The controller map was derived from the static value function and we used multilinear interpolation again in order to obtain a continuous controller. We validate our controller by simulating the controller performance on our stochastic system model and evaluating it versus the recorded operator runs according to our running cost function defined in the Value Iteration. In summary the resulting controller outperforms the operator on average and in the worst case has comparable performance to the operator from 1000 simulation runs. Improving the controller performance further would be possible by using a more accurate system model or using a different grid parameterization than the one we used for computational efficiency reasons.

Electric vehicles are a fast-growing market in the automotive sector. In addition, the widespread use of renewable energy to power electric vehicles makes them sustainable, with considerably low greenhouse gas emissions. As a result, service providers are switching to fleets of electric vehicles to promote environmental sustainability. However, unlike conventional vehicles, EVs require unique infrastructure to charge them. This leads to some technical and economic challenges. Therefore, intelligent charging strategies are needed to charge EV fleets optimally.

The thesis primarily focuses on minimizing the energy and battery degradation costs for a fleet operator using different charging strategies. To accomplish this objective, a joint optimization technique is used to solve the problem. The method used is an optimal exchange problem that works by clearing market constraints. Specifically, an ADMM-based distributed charging problem is used for charging the EV fleet. The algorithm is implemented for different charger power levels for the different strategies to analyze the difference in energy and battery degradation costs. Furthermore, a variable charger allocation method is proposed to charge the EV fleet.

Natural Language Processing (NLP) deals with understanding and processing human text by any computer software. There are several network architectures in the fields of deep learning and artificial intelligence that are used for NLP. Deep learning techniques like recurrent neural networks and feed-forward neural networks are used to develop language models that perform several NLP tasks. Over the years, researchers have worked on developing state-of-the-art language models that achieve high accuracy and performance for NLP applications. With the development of deep neural network language models, the computational resources requirements and the energy costs for training and running language models increased. This led to research to compress the language models, thereby reducing the computational complexity of the language models. One of the methods used for this is tensor decomposition, like the tensor-train (TT) decomposition. During this thesis work, the application of the TT-decomposition method for compressing the embedding layer in a long-short-term memory model was investigated. Specifically, the effect of factorization and the order of factors in the embedding layer when it is represented in the TT-matrix format on the maximum test accuracy of the long- short term memory model for the NLP task of sentiment analysis was investigated. This was done by considering three different factorizations of the embedding layer in the model. Further, the effect of change in TT-ranks (hyperparameters of the model when the embedding layer is represented in the TT-matrix format) on the maximum test accuracy was also investigated. Based on the investigation and empirical results obtained, this thesis concludes that by having a larger number of factors in the factorization of the embedding layer, the maximum test accuracy of the model increases. Further, in a particular factorization, when the factors were arranged in such a way that the maximum values of the TT-ranks had a smaller gap, the maximum test accuracy of the model improved. In one particular configuration of the model, the number of parameters was reduced by 24.5 times compared to that of the original uncompressed model, and a maximum test accuracy of 77.10% was achieved compared to a maximum test accuracy of 78.05% in the case of the original model.

Industrial control systems (ICSs) are used widely throughout the world for the control of large, complex industrial plants and consist of the entire setups of control system including sensors, PLCs (Programmable Logic Controllers), actuators and communication devices. The communication between these ICS devices is performed using industrial communication protocols such as Modbus, EtherCAT, etc. With the advancements in the use of the internet, ICS are being enabled to share real-time information over the internet worldwide. While these features make the ICS more accessible for remote supervisory control, they also make them vulnerable to cyber-attacks. This makes it the need of the hour to investigate risks and impacts of cyber attacks on ICS. Generating, injecting and testing cyber attacks on real world ICS, controlling critical infrastructure will involve several financial risks and safety issues. This gives rise to the necessity of an ICS testbed, with the ability to inject and test cyber attacks, in a safe and secure environment. A testbed provides a cheaper alternative for testing impacts of cyber attacks and also offers more flexibility to simulate multiple industrial scenarios. Together, these aspects form the core reasons behind the requirement of an ICS testbed. As the real world ICSs are often costly and specialized for industrial usage, there are not many research laboratories around the world with the availability of a testbed to study cyber attacks. Therefore, this Master of science thesis, through its primary research question, investigates if a cyber-attack testbed can be built in the NERD lab at the Delft University of Technology, which is able to replicate a real-world ICS network to identify and test vulnerabilities of ICSs working on Modbus protocol to cyber-attacks? To answer this question, this report studies the vulnerabilities in ICS working with Modbus protocol. A novel design for an ICS testbed for generating and testing cyber attacks at NERD Lab in TU Delft is presented during this thesis. The proposed testbed utilizes real world ICS components such as PLC and HMI, combined with a plant simulator, which is used for simulating an industrial process. The testbed utilizes a Linux based attack PC to generate and inject various cyber attacks. A virtualization platform connects the attack PC to the ICS network, giving the flexibility of injecting attacks on the testbed, without the attacker being physically present on the plant site. With the use of real world ICS, the testbed therefore allows to replicate a typical ICS scenario in the real world industry. Further, a simulated version of the actual testbed, with open source softwares, mimicking the ICS systems has been developed in this report. This simulated version provides a cheaper and flexible platform to perform initial testing on the working of the testbed and checks the feasibility of the actual testbed. The testbed simulates a plant, controller, and HMI in Matlab/Simulink on different physical PCs, which communicate with the Modbus protocol. An attack PC with a virtualization environment has been used to launch cyber attacks on the simulated testbed, same as that to be used in the proposed testbed at Delft University of Technology. Two main types of cyber attacks namely, Man-In-The-Middle (MITM) and Denial-of-Service (DoS) attack have been successfully implemented on this simulated testbed. To conclude this thesis, advanced versions of these attacks have also been developed and their impacts have been analysed.

For this thesis we have compared communication times of different control schemes on a wireless control network for a water irrigation system. Therefore a control application which can run different protocols was created. We have shown that a significant reduction of communication time can be achieved by using event-triggered control (ETC), and by using self-triggering techniques we could reduce this even further. Wireless control networks (WCN) are networks of one or multiple controllers, sensors and actuators which share a digital communication network. Sensors send their measurements to the controller, and the controller sends a control signal to the actuators. Because the nodes are digital devices, periodic control is the standard approach. Communication therefore is not needed all the time, and nodes can sleep in between updates to save energy. Event-triggered control is a technique to further reduce communication of resource constrained nodes in the network by only updating control when an event, typically a significant change since the last control update, occurs. Water irrigation systems (WIS) are networks of channels transporting water from main rivers, to smaller rivers which need to be controlled to avoid losses. At TU Delft a model of a water irrigation system with three pools has been connected to a WCN to simulate this. To create an application for this testbed, supporting different control schemes, a low power embedded OS capable of doing multitasking with a replaceable network stack is needed. The Contiki Operation System is capable to do this and has been used by the D3S Research Group of the University of Trento to build wireless control bus (WCB), and has also been used for the WIS control application. WCB uses network floods from a protocol called Glossy and a schedule to synchronise communication with low power, and supports both periodic and event-triggered control. Experiments with WCB show periodic control is possible with only 7% communication time for this setup, and event-triggered control with only 2% without much loss of performance. Because the testbed unit has some technical problems, a hardware-in-the-loop simulation was created to connect to the WCN to do the experiments. A further reduction in communication has been achieved by combining a predictive control method called PSTC with ETC, which we call ETC+. Advantages over normal ETC are not large in our experiments, and there are still practical problems, but they show this is a promising research area.

Grasping objects in a smooth humanlike motion, instead of the more typical pick-and-place approach, includes multiple aspects that need to be performed correctly for a successful grasp. These aspects involve moving the end-effector such that its surface makes and retains contact with the object while also coordinating the movement of the gripper to securely grasp the object. This work investigates how the intricate task of grasping may be learned from humans based on kinesthetic demonstrations. Due to the complexity of the task, these demonstrations are often slow and even slightly flawed, particularly at moments when multiple aspects (i.e. end-effector movement, orientation and gripper width) have to be demonstrated at once. Rather than training a person to provide faster demonstrations, non-expert users are provided with the ability to interactively modify the dynamics of their initial demonstration through teleoperated corrective feedback. This in turn allows them to teach motions outside of their own physical capabilities. In the end, the goal is to obtain a faster but reliable execution of the task.
The presented framework learns the desired movement dynamics based on the current Cartesian position with Gaussian Processes, resulting in a reactive, time-invariant policy. Using Gaussian Processes also allows online interactive corrections and active disturbance rejection through epistemic uncertainty minimization.
The experimental evaluation of the framework is carried out on a Franka-Emika Panda. Tests were performed to determine i) the framework's effectiveness in successfully learning how to grasp an object quickly, ii) ease of policy correction to environmental changes (i.e. different object shapes and mass), and iii) the framework’s usability for non-expert users.

With modern multi-function radars becoming more flexible, handling the limited amount of resources of these radars becomes increasingly important. In this thesis the radar resource management (RRM) problem in a multi-target tracking scenario is considered. Partially observable Markov decision processes (POMDPs) are used to describe each tracking task. By comparing the future effect of radar actions using model predictive control (MPC), the POMDPs are solved in a non-myopic way. The model predictive control problem can be decoupled into sub-problems using Lagrangian Relaxation to reduce the computational complexity of the solution method. An algorithm based on golden section search is employed to find the Lagrange multiplier. An interacting multiple model filter is used to allow the method to be effective in RRM problems involving the tracking of targets performing a broad number of maneuvers.
The novel approach is compared to an existing solution method based on policy rollout and Monte Carlo sampling. Through simulations of dynamic multi-target tracking scenarios in which the cost and computational complexity of different approaches are compared, it was shown that the computational complexity is greatly reduced while the resulting resource allocation results remain similar.

This MSc thesis presents the development of a viewpoint optimization framework to face the problem of detecting occluded fruits in autonomous harvesting. A Deep Reinforcement Learning (DRL) algorithm is developed in order to train a robotic manipulator to navigate to occlusion-free viewpoints of the tomato-target. Two Convolutional Neural Networks (CNN), You Only Look Once (YOLO) version 3 and Mask Regional Convolutional Neural Network (Mask R-CNN), are trained and evaluated in order to obtain visual information from the tomatoes. The two trained CNN achieve high detection accuracy and surpass other fruit detection methods. The instance segmentation Mask R-CNN result and an image processing algorithm are applied in an occlusion modelling method. The vision-based reward formulation for the DRL algorithm is closely related to the occlusion modelling metric. The DRL algorithm is trained and evaluated in a simulation environment by mounting a camera on a robotic arm. The use of different reward scheme and exploration strategy during DRL training shows their effect on training performance. The DRL training performance is assessed by illustrating the maximum visible tomato-target fraction per episode, the steps needed per episode, the trajectory of the robot's end-effector, and the initial and final tomato-target viewpoint in each episode. The evaluation results show a satisfactory performance, which depends on the reward and exploration strategy, as in the majority of the cases the robot can fully see the tomato-target after a few steps.

Modern times have seen an increasing use of networked control systems, in which plants and controllers may not necessarily have a direct link but instead be connected through a network, thereby closing control loops over multiple nodes.

The system may also be spread out spatially over a large area, and thus the associated network delays could greatly hamper control performance, potentially affecting the closed-loop stability of the system. In such scenarios, event-triggered control approaches could greatly reduce network congestion
by allowing a means for the controllers to send control loop computation packets over the network only when required, in an event-driven manner, rather than through periodic transmissions. However, in practice, the number of parallel channels is limited compared to the number of controllers and hence the transmission of packets needs to be scheduled carefully to avoid network conflicts.

This thesis explores using a network of timed (game) automata composed of models representing a networked control system’s control loops and its communication network. This reduces the scheduling problem of transmission of control loop computations to one of creating strategies using known algorithms, with the objective being to avoid network conflicts brought about by simultaneous transmissions. Furthermore, the proposed automata models also aim to reduce the conservatism of generated scheduling strategies by allowing the control loops a bounded number of retransmission attempts to send packets over the network in case it is already occupied. The concept is finally demonstrated in practice using simulated plants and controllers distributed over multiple machines connected via a physical CAN network.

Interactive machine learning describes a collection of methodologies in which a human user actively participates in a novice agent’s learning process, through providing corrective or evaluate feedback or demonstrative actions. A primary assumption in these methods is that user input is at worst nearoptimal, however a realistic set of demonstrations will often contain conflicting or poor examples, which degrade the quality of the learnt policies. This project explores methods for the detection of such undesirable features in data and develops an algorithm for policy training with suboptimal demonstrations, while leveraging the generalisation and scalability qualities of artificial neural networks. Uncertainty estimation, which presents a structured approach for the quantification of a network’s confidence in the accuracy of its output, based on the observed training data, is applied for the detection of unwanted features in a demonstration dataset. The particular focus of this project is conflicting data resulting from scenarios with equivalent action choices, such as the obstacle avoidance setting. Following thorough testing on various environments, it is shown that novice policies may be trained to achieve a desired goal in multi-dimensional spaces with either discrete or continuous data, despite the presence of conflicts in this training data.