Autonomous vehicles offer significant potential for improving traffic efficiency and reducing
fuel consumption, with Model Predictive Control (MPC) being widely used due to its ability
to guarantee constraint satisfaction and safety while providing optimal control performance.
However, car models traditionally used in MPC approaches for vehicle control often overlooks
discrete dynamics like gear changes, which are critical for optimizing vehicle fuel consumption. Advancements have incorporated these discrete dynamics into MPC, resulting in a hybrid model that considers both continuous and discrete dynamics. The incorporation of
the fuel model, along with these discrete dynamics, significantly increases the computational
complexity of the MPC problem, making real-time implementation challenging. To address
this issue, Reinforcement Learning (RL) can be leveraged to simplify the optimization problem by learning policies that determine key discrete components, such as gear selection. This
allows the MPC controller to handle a simpler optimization problem, thereby reducing the
computational burden and enabling real-time control. This research aims to propose a new
approach to integrate RL and MPC for vehicle control, where RL is used to manage gear
transitions and MPC controls the overall vehicle dynamics, offering a computationally efficient solution, while achieving near optimal performance comparable to the conventional
MPC approach.

Energy systems have been continuously evolving with the advancement in technology. The expected result would be a smooth transition towards clean and more sustainable energy systems which work closely with one another. The focus on using the available energy resources optimally and effectively is a need for energy providers as well as the energy consumers. However, there are different challenges in making these optimal decisions because most of the energy providers have their independent networks. The progress in energy sector has promoted the use of integrated energy systems from different geographical locations. Integrated systems include several large subsystems called areas consisting of electricity and natural gas networks. These large systems are connected to one another through connections known as tie-lines and/or tie-pipes and the energy dispatch can be controlled by the area operator. The main intention of an integrated system is that the electricity and natural gas networks are closely linked as opposed to the formerly isolated systems.
The goal of an optimal energy flow problem for an IEGS network consisting of an objective function is to minimize the system's overall operational cost while satisfying the constraints for the electricity network, natural gas network and the coupling constraints. This study attempts to provide insights into some of the available methods of relaxation/approximation for linearizing the nonlinear, non-convex natural gas flow constraints, their performance, efficiency, and reliability of these methods for multi-area networks. In this overview, various formulations and solution methods of the optimization problem have been examined. Their performance has been compared based on certain performance metrics to find the best possible method to solve such complex optimization problems. Since the integrated systems usually are extremely large, it is vital to separate them into smaller subsystems to solve the optimization problem efficiently. There are different methods used for solving these decentralized optimization problems and it is important to study these algorithms for multi-area IEGS.
The investigation begins with an introduction of the research problem to find the best possible optimization method for IEGS. This study emphasizes the significance of relaxation techniques used to relax the non-convexity and studying optimization methods for integrated electricity natural gas systems. The mathematical formulations and optimization strategies utilized to solve the models are explained in detail. The performance of optimization algorithms is assessed based on certain performance metrics. Finally, this thesis concludes by summarizing the primary findings for the best possible optimization method and suggesting future research possibilities in this topic. To determine which approach best fits the OEF problem for IEGS, a thorough numerical comparison of various relaxation techniques, centralized and decentralized scheme of operation must be made. It is imperative to note that these techniques solve the approximated problem, not the original nonlinear, non-convex problem, which would leave some room for errors and hence an area of future improvement.

For trajectory prediction within autonomous vehicle planning and control, conditional variational autoencoders (CVAEs) have shown promise in accurate and diverse modeling of agent behaviors. Besides accuracy, explainability is also crucial for the safety and acceptance of learning-based autonomous systems, especially in autonomous driving. However, the latent distributions learned by CVAE models are often implicit and thus possess low explainability. To address this, we propose a semi-supervised generative modeling framework, \textbf{\textit{PrefCVAE}}, which utilizes partially and weakly labelled preference pairs to imbue the CVAE's latent representation with semantic meaning. This approach enables the system to estimate measurable attributes of the agents, and to generate manipulable predictions under the CVAE framework. Results show that incorporating our preference loss allows a CVAE-based model to make conditional predictions using the semantic factor of prediction average velocity. Our augmented framework also does not significantly degrade the baseline accuracy of prediction. Additionally, we show that the latent values learned with PrefCVAE better represent the semantic information contained in the data. Finally, we discuss the potential of this loss design to extend to other machine learning applications beyond trajectory prediction, as well as essential tricks for adaptation of human labeling. We hope that our empirical study offers the broader representation learning community a fresh perspective on inductive bias for disentangled and explainable latent representations in deep generative models. Specifically, we demonstrate that preference pair supervision, a simple and cost-effective approach, can effectively aid in learning semantic meanings for sampling-based generative models like the CVAE.

Over the years, the conventional open-loop basal-bolus regiment has proven to be inadequate for long-term glucose management of Type-1 diabetes mellitus (T1DM) patients. The ’artificial pancreas (AP)’, which is characterized by a closed-loop control system that typically relies solely on subcutaneous glucose measurements as a feedback to deliver insulin corrections, is widely regarded as a viable alternative to the former approach. An earlier literature survey showed that model predictive control (MPC) and proportional-integral-derivative (PID) control are the two most commonly employed approaches in APs to treat T1DM, especially the former in recent years, due to its ability to incorporate flexible safety-critical constraints and account for inherent system delays. With most state-of-the art MPC controllers designed to exert direct control on insulin corrections over the patient however, the extent to which closed-loop performance can be maximised greatly depends on how accurately the plant is modelled. In addition to the immense difficulty in identifying certain physical parameters and non-linearities that describe an individual patient’s insulin-glucose pharmacokinetics, practical challenges arising from the collection of open-loop data from the patient hinder the identification process of a prediction model. The resulting model-plant mismatch is detrimental to the closed-loop performance, especially during postprandial periods when hypoglycaemic episodes are most likely to occur. As a countermeasure for the model-plant mismatch, insulin delivery rates are often tightly enforced, which may increase robustness of the MPC controller but comes at a price of reduced, sub-optimal performance due to worse-case conservative assumptions.

This thesis presents a dual-layer control scheme: a lower layer comprising of a control architecture that regulates the rate of insulin delivery for corrections against disturbances, and an upper layer that represents a performance-driven tuning framework using Bayesian Optimisation (BO). The proposed control architecture is hierarchical, consisting of an inner-loop PID controller, and an MPC which serves as the outer-loop controller that governs the reference input signal to the inner-loop control system. The upper layer tuning framework is implemented in two stages, the first on a set of clinical-related parameters, and the second a controller and model parameter, as part of the individualized control objective. In both stages, the BO framework optimises the parameters via a daily query of one parameter set and its corresponding performance value, which is computed based on experimental data. The proposed control scheme is designed and evaluated using a cohort of 10 adult virtual patients of the U.S. Food & Drug Administration (FDA)-accepted University of Virginia/Padova simulator through several in silico scenarios. Numerical evaluation and results show that the proposed hierarchical controller outperforms its baseline counterpart, and that glycaemic performance of all the tested subjects improved using the proposed tuning approach compared to under nominal settings. Notwithstanding the absence of a probabilistic safety-critical constraint in the BO problem, the employment of a barrier-like penalty in the objective function, subjected to a deterministic domain-based constraint that is augmented using a rule-based approach, demonstrated to be an efficient safeguard against hypoglycaemia during the parameter tuning process.

Autonomous motion planning requires the ability to safely reason about learned trajectory predictors, particularly in settings where an agent can influence other agents' behavior. These learned predictors are essential for anticipating the future states of uncontrollable agents, whose decision-making process can be difficult to model analytically. Thus, uncertainty quantification of these predictors is crucial for ensuring safe planning and control. In this work, we introduce a framework for interactive motion planning in unknown dynamic environments with probabilistic safety assurances. We adapt a model predictive controller (MPC) to distribution shifts in learned trajectory predictors when other agents react to the ego agent's plan. Our approach leverages tools from conformal prediction (CP) to detect when the other agent's behavior deviates from the training distribution and employs robust CP to quantify the uncertainty in trajectory predictions during these agent interactions. We propose a method for estimating interaction-induced distribution shifts during runtime and the Huber quantile for enhanced outlier detection. Using a KL divergence ambiguity set that upper bounds the distribution shift, our method constructs prediction regions with probabilistic assurances in the presence of distribution shifts caused by interactions with the ego agent. We evaluate our framework in interactive scenarios involving navigation around autonomous vehicles in the BITS simulator, demonstrating enhanced safety and reduced conservatism.

Modern control systems require control strategies that can handle multiple levels of abstraction while providing formal guarantees of system behavior. Traditional hierarchical control approaches often lack formal interfaces between different layers of abstraction, making it challenging to ensure system-wide properties and extend implementations across different platforms.

This thesis presents a contract-based hierarchical control framework that provides formal guarantees across different layers of abstraction. Building upon recent developments in vertical contracts with heterogeneous refinements, we develop a three-layer control architecture for robot surveillance tasks, decomposing the problem into motion planning, trajectory generation, and tracking control. Each layer operates at its own time scale and uses appropriate model abstractions, with contracts ensuring system-wide guarantees while vertical system relationships enable proper interfacing between layers.

We validate our framework through both simulation studies and physical implementation on Elisa3 robots. The simulation results demonstrate the framework's ability to satisfy high-level temporal logic specifications while maintaining guarantees at each layer. The physical implementation reveals practical challenges in transitioning from simulation to hardware, leading to adaptations in the control architecture that maintain theoretical guarantees while accommodating hardware constraints.

Artificial Neural Networks (ANNs) have emerged as a powerful tool for classification tasks due to their ability to outperform traditional methods. Nevertheless, their effectiveness relies heavily on the availability of large, varied, and labeled datasets, which are often not available. To counter this constraint, data augmentation techniques have emerged, leveraging existing data to generate additional, variant data. Extending these techniques to multi-dimensional time series data, such as the transportation mode detection data considered in this thesis, however, introduces challenges. In response, generative models such as Variational Autoencoders (VAEs) have shown promising advancements.

In this context, this thesis investigates the application of the Iterative Hierarchical Data Aug- mentation (IHDA) algorithm for ANNs, which represents a VAE-based data augmentation technique. The IHDA method utilizes VAEs not only to generate new data samples but also to map existing data to a lower-dimensional latent space, which is then utilized for identifying samples that might require additional training. The proponents of this method, Khan and Fraz, reported an accuracy elevation for the considered transportation mode detection classifier from 83% to 92%. However, due to the absence of publicly accessible code for this algorithm, the initial step of this thesis involved implementing the IHDA algorithm. Further, this research proposed and incorporated advancements like the σ-VAE, aimed to improve the generative capacity of the VAE and to refine its latent space mapping. Additionally, the Kullback-Leibler (KL) divergence was introduced as a similarity metric, aiming to optimize the identification process of samples that require retraining.

Unfortunately, the results reported by Khan and Fraz could not be reproduced in this study. Furthermore, despite the potential shown by the σ-VAE to improve the generative capacity and refine the latent space mapping, along with the enhanced sample identification through the KL divergence, these enhancements did not lead to an overall improvement in the IHDA algorithm. This was primarily attributed to the low generative performance of the VAEs utilized, which also hindered a thorough evaluation of the effectiveness of the IHDA algorithm.

Given these outcomes, it is suggested that future work should focus on employing more complex VAE models with the potential to enhance their generative performance, which, in turn, could improve the IHDA algorithm’s overall effectiveness.

Researchers have been interested in studying the connection between emotion and memory for decades but much remains unknown due to the elusive nature of the human brain. Furthering our understanding of the phenomenon is crucial for improving the treatment of neurological disorders associated with emotion dysregulation, as well as for enhancing autonomous systems that interact with humans, such as robotic prostheses and artificial intelligence. A promising approach to studying cognitive processes is developing computational models of brain activity, which have the potential to uncover the underlying neural mechanisms.
This master's thesis presents a novel dynamical model of the neural activity underpinning the emotional associative memory encoding process. This type of memory is especially complex and poorly understood as it requires memorizing the relationships between various environmental stimuli that may elicit distinct emotions. To model this phenomenon, the thesis proposes a network model using the Dynamic Causal Modeling (DCM) framework, focusing on the Amygdala (Amy), the Hippocampus (Hip), and the Orbitofrontal Cortex (OFC) due to their central roles in emotional associative memory. Furthermore, the thesis provides an analysis of network state coordination and state error stability, deriving analytical bounds for state error dynamics that illustrate how the model's properties are linked to improved memory encoding.
The dataset used to estimate the DCM model comes from a functional Magnetic Resonance Imaging (fMRI) study conducted by Zhu et al. at Donders Institute for Brain, Cognition and Behaviour, where participants were asked to memorize pairs of two emotionally potent images (group EE), two neutral images (group NN), or one neutral and one emotional image (group NE). The DCM model developed in this thesis reflects how the neural dynamics differ among the groups and the modeled brain regions, corroborating many of the behavioral results of the experiment and bringing novel insights. Based on stability and error dynamics analysis, the model of group NE is found to exhibit the best overall information flow, which is consistent with the experiment, where this group achieved the best memorization results. Furthermore, the coupling between Amy-Hip is shown to play a key role in improving memory encoding.
Another important contribution of this thesis is the extensive overview of DCM in Chapter 2, which discusses the mathematical foundations, the system identification algorithm, criticism raised against the method, studies of its statistical validity, and advice on implementation. Additionally, Appendix B presents the first guide for DCM script development in the SPM12 toolbox in MATLAB. Furthermore, this thesis compares the performance of four different variations of DCM, proving that Stochastic DCM achieves superior results in experimental paradigms with brief emotional stimuli. The success of this approach is believed to stem from its capacity to handle model misspecification and neuronal noise. Overall, the Stochastic DCM developed in this thesis is considered to provide the most comprehensive representation of neural dynamics governing the encoding of emotional associative memory compared to other existing dynamical models.

Sorting systems form an example of event driven systems. These types of systems are referred to as discrete event systems (DES), and they consist of jobs that need to be performed at available resources. In an autonomous sorting system, jobs consist of robots receiving and delivering parcels at the correct locations. With scheduling, optimal allocation of the jobs to those resources over time is computed, where the decisions that need to be made are routing, ordering and synchronization. The behaviour of DES is often described by non-linear models, but max-plus linear (MPL) systems are a class of DES that can be described by a model that is linear in the max-plus algebra. This algebra uses two operators maximization and addition. Allowing different routes and switching between orders of jobs extends an MPL system to a switching max-plus linear (SMPL) system. Robots in a sorting system often have many routes to choose from, and need to make order choices with respect to other robots in the system.

In this thesis, a general SMPL model is made for the autonomous sorting system at software company Prime Vision, which can be applied to any sorting area design. The solution to the scheduling problem for the model results in a time schedule for the active robots at the correct locations in the sorting area, as well as the optimal decisions on routing, ordering and synchronization. The optimization problem is solved with a model predictive scheduling (MPS) approach and recast as a mixed integer linear programming (MILP) problem. The model is created in Python and the optimization problem is solved with Gurobi. The resulting schedule is visualized with a simulation, in which the decisions of the robots are clearly shown. An idea for implementation of the optimization into the sorting system is given as well.

Over nine billion people will have to be fed fresh and healthy food in 2050 according to United Nations. This puts pressure on the horticulture sector, responsible for a large portion of the world’s food production. The literature has shown that automatic optimal control algorithms are able to make better use of resources and can even outperform growers in terms of resource efficiency.

While applications of automatic optimal control methods show promising results, few are made adaptive to the greenhouse-crop system, assuming the system to be time-invariant. That is, the prediction models or techniques used in the controllers are not updated to adapt better to the evolving greenhouse-crop system. This thesis sets a first step towards this on-line system adaptation: it investigates the possibility to use a linear state-space model to estimate the non-linear greenhouse-crop systems dynamics around an operating point, for the purpose of automatic optimal climate control. In this work, a linear state-space model is identified and is implemented in an automatic optimal controller based on Model Predictive Control (MPC). Moreover, a greenhouse-crop simulation is built from state-of-the-art models based on first-principles. This ground-truth simulation is used to generate data and test the proposed control method.

Socially compliant robot navigation in pedestrian environments remains challenging owing to uncertainty in human behavior and varying pedestrian preferences in different social contexts. Local optimization planners like Model Predictive Control can incorporate collision avoidance constraints, but they can only lead to socially compliant trajectories if the cost function embeds information about the desired social behavior. The same holds for Reinforcement Learning, where a sophisticated reward function needs to be defined. However, formalizing social behavior through a reward or cost function is difficult due to the complex nature of pedestrian behavior. Imitation learning allows for inferring the desired behavior by learning from human demonstrations, making them suitable for learning socially compliant navigation policies but without any safety considerations. In this work, we propose to learn a socially compliant navigation policy directly by observing surrounding pedestrians’ trajectories from a commonly available detection and tracking pipeline and combine it with a local optimization planner to enhance safety. A Subgoal Recommender policy is developed to guide the local optimization planner to generate socially compliant trajectories by providing intermediate subgoals. To adapt the policy to changing social contexts without forgetting previously learned information, we train the Subgoal Recommender in a Continual Learning setup exploiting new pedestrian data.
We demonstrate in simulation that our method can learn a policy that has similar performance metrics as that of the observed trajectories with 95% confidence estimated from a t-test, resulting in a lesser number of collisions. Further, the policy can adapt to different social preferences exhibited by pedestrians, while being able to remember the learned behavior in a previously encountered social context. Furthermore, we show that our proposed method can learn navigation policies from actual pedestrian data recorded using the onboard perception pipeline of a Clearpath Jackal robot.

Designing controllers for complex robots is not an easy task. Often, researchers hand-tune controllers for humanoid robots, but this is a time-consuming approach that yields a single controller which cannot generalize well to varied tasks. This thesis presents a method which uses the NSGA-II multi-objective optimization (MOO) algorithm with various training trajectories to output a diverse Pareto set of well-functioning controller weights and gains. The best of these are shown to also work well on the real Talos robot. The learned Pareto front is then used in a Bayesian optimization (BO) algorithm both as a search space and as a source of prior information in the initial mean estimate. This learning approach, which combines the two optimization methods, is capable of finding a suitable parameter set for a new trajectory within 20 trials and outperforms both BO in the continuous parameter search space and random search along the Pareto front. The few trials required for this formulation of BO suggest that it could feasibly be applied on the physical robot using a Pareto front generated in simulation.

The platooning of trucks provides significant benefits in the existing transportation systems regarding social, economic and environmental aspects. Due to the platoon's high dependence on sensors, it is critically important to have fault-tolerant countermeasures that deal efficiently with failures by achieving minimal risk condition. This thesis tackles designing an emergency escape manoeuvre for a faulty truck that has lost its ability to monitor the driving environment while moving in a platoon formation on a highway. A fallback strategy is proposed that leads the faulty truck to park on the shoulder of the road and allows the rest of the platoon participants to continue their journey. To solve this problem, first, a functional platooning system should be designed. Therefore, a nonlinear bicycle dynamic model is employed, while a new Bidirectional (BD) Cooperative Adaptive Cruise Control (CACC) system is utilized for control purposes. The emergency escape manoeuvre has to be pre-computed and ready to be triggered when the failure occurs, as concerning defined specifications, limitations and constraints. It is generated using a quintic splines methodology, while a new optimization approach regarding the minimization of jerk peaks is adopted, enhancing the efficiency of the design procedure. Furthermore, a gap-closing controller is employed and tuned accordingly to reform the platoon after the faulty truck abandons the formation. Moreover, a suitable decision logic is defined to achieve smooth transition among regular performance and fallback state at both platoon and inter-vehicle level. The functionality of each component is firstly evaluated individually, based on specific performance metrics. Finally, all the mentioned components are put together and the fallback strategy that executes emergency escape manoeuvre in platoon context is derived. Several simulation tests illustrate proof of principle for the proposed framework. It is shown that the proposed strategy indeed achieves minimal risk condition for the faulty truck achieving minimal tracking errors. A global applied strategy is adopted for the gap-closing controller by making healthy follower(s) travel with the maximum possible acceleration. At the same time, the leading truck of the formation maintains a constant speed until the time gap between them becomes $0.8s$, before the switch back to the CACC, to complete this action as fast as possible and safety issues in the driving performance. Additionally, the CACC system is evaluated to ensure that the string stability criterion is satisfied in the mode of operation. Moreover, a virtual leader mechanism is introduced to fill in faulty truck's blind spot in the road that arise from the sensor failure. It is shown that it can provide a sufficient window of movement for the platoon before switching to the fallback strategy.

Stability, safety and optimality are often sought-after properties in the field of controller synthesis. In the last century, linear control theory has matured to a level where scalable algorithms are widely available that are able to synthesize controllers with stability and optimality guarantee. However, the synthesis of safe controllers largely remains an open question. Furthermore, when nonlinear systems are considered, these methods often result in sub-optimal performance, or fail to provide a stabilizing solution at all. Current nonlinear controller synthesis methods typically lack stability, safety and/or optimality guarantee, or require computationally expensive online optimization. This thesis presents a novel method that is able to overcome these limitations. The presented controller synthesis method combines techniques from Approximate Dynamic Programming, Lyapunov theory and barrier theory, to synthesize an optimal controller in the form of a Neural Network. Alongside the controller a second Neural Network is deployed, which is trained to serve as a certificate function to provide stability and/or safety guarantee. To assure the correctness of the procedure, Satisfiability Modulo Theory and Counterexample-Guided Inductive Synthesis are utilized. We subject the method to a number of case studies, through which the versatility of the method is demonstrated. We show the method is able to work with linear and nonlinear systems, as well as systems with input and state constraints. Furthermore, we show the method yields controllers that are able to outperform linear controllers in terms of cost minimization.

With the rapidly growing world population and improving living standards of upcoming economies, the demand for fresh food is increasing vastly. Greenhouses have proven to be very effective in increasing crop yield since they offer a sheltered environment that can be controlled. Greenhouse growers need to take care of many processes, among which are maximising the crop yield while using as little resources as possible and controlling the indoor climate. An approach to alleviating demand for experienced growers is to automate processes within the greenhouse that aid in setpoint generation and setpoint tracking. Existing literature has presented multi-level architectures to cope with the two different timescales on which the two greenhouse subsystems act, i.e., the crop and greenhouse indoor climate subsystem. Current approaches in literature employ model-based techniques, which need tedious calibration
per considered situation due to the influences of the local weather, environment and highly non-linear underlying physical processes. Therefore, these model-based approaches are scarcely used in the horticultural sector. Thus, this thesis proposes a data-driven control scheme that deals with long-term crop production control while taking into account resource usage.

The proposed control scheme uses temporal and functional decomposition of the overall problem to accommodate the different timescales on which the greenhouse indoor climate subsystem and crop subsystem act. With the focus on the entire growing season, this thesis emphasises the importance of the climate strategy for crop production control. The aim of this thesis is threefold. Firstly, a non-linear simulator model for the crop dynamics has
been selected, adapted, and calibrated, using the Autonomous greenhouse challenge (AGC) dataset. Secondly, the overall crop production control problem has been decomposed into a hierarchical structure, including two subproblems with different objectives, acting on different timescales and using different system representations. The communication protocol between the two layers has been determined, and the setpoint tracking controller is established for the lower layer and tested on a benchmark temperature reference trajectory. Thirdly, the Data-enabled
predictive control (DeePC) algorithm is leveraged for the crop production control problem and compared in simulation against a conventional Economic model predictive control (EMPC) setpoint generating controller. Both use the aforementioned established setpoint tracking controller for controlling the greenhouse towards the generated climate strategies.

In simulation, it is shown that the DeePC setpoint generating controller results in a climate strategy that results in a lower crop yield by 1.51% when compared to the EMPC climate strategy and 0.54% less when compared to the predefined benchmark reference trajectory. Furthermore, the resource usage needed for the climate strategy proposed by the DeePC setpoint generating controller uses 1.75% more heating resources when compared to the EMPC setpoint generating controller and 11.1% more when compared to a predefined temperature setpoint trajectory. The DeePC climate strategy results in 7.92% and 5.27% less electricity use when compared to the EMPC and predefined benchmark climate strategy, respectively. The DeePC climate strategy CO2 usage is 11.4% less when compared to the EMPC climate strategy CO2 usage and 20.8% less when compared to the predefined benchmark climate strategy. The decrease in resource usage compensates for the lower crop yield of the DeePC climate
strategy. According to the metric of net economic profit, the DeePC generated climate strategy outperforms the benchmark climate strategy and EMPC climate strategy.