Rapid and continuous advances in communications and computer technology are spurring
a host of new concepts in road traffic control. From simple traffic control measures like
lane segregation for wheeled and pedestrian traffic in the 14th century to the use of artificial
intelligence to control traffic, we have come a long way. The advancement of technology in
the automotive sector along with the industrial revolution led to the number of vehicle owners
seeing a drastic increase in the 20th century. There was a growing need for infrastructure to
handle this new volume of vehicles which led to the construction of several road networks in
both urban and freeway settings. Adding infrastructure was a valid method to handle traffic
problems until we realized that each new addition required a significant amount of land that
was slowly reducing. This brought along the need for newer and more efficient traffic control
techniques due to increased vehicles on the road, the different kinds of vehicles itself and the
need to reduce construction of roads as a means of reducing traffic.
In this thesis report, we discuss the different control methods using reinforcement learning
to tackle the freeway traffic control problem. The thesis covers the fundamentals of freeway
traffic control, reinforcement learning and the agents used for control. It focuses on the
creation of the freeway network, environment setup for reinforcement learning application
and the choice of agents mainly SAC in order to implement continuous actions to improve
combined ramp metering and variable speed limit control in more complex scenarios

To make interactions between humans and robots safer, soft robots may offer a solution. The autonomous closed loop control of these robots so far, however, is not accurate enough to perform specific tasks as handovers. The purpose of this paper is to propose a control algorithm that can make the control of soft robots accurate enough for human-to-robot handovers. The main focus within this research was on the state estimation and the Jacobian based control. Due to gravity, the internal system state is not an accurate representation of the actual system behavior. The optimized pose estimation solves this problem. In order to test the proposed algorithm, the complete control architecture has been implemented including the object and end-effector detection. Experiments have shown that the algorithm works with different step sizes within the Jacobian based control,
consistently resulting in a successful handover. A second experiment has shown that the handovers are still successful and faster when a human guides the robot toward the right position. Lastly, a possible use case has been shown.

Traffic congestion remains a critical challenge with profound economic, safety, and environmental implications, exacerbated by the continuous growth of global population and vehicle densities. The expansion of road infrastructure is often impractical due to excessive costs and spatial constraints. In response, intelligent traffic management strategies have gained prominence, leveraging advanced control methodologies to optimize flow and mitigate congestion. Among these, Model Predictive Control (MPC) has been extensively applied due to its capacity to handle complex, constrained systems. However, the accuracy of the underlying prediction models inherently determines its effectiveness. Traditional system identification methods can have inherent uncertainties and are often only accurate to a certain confidence level, leading to potential performance degradation in dynamic traffic conditions. To overcome this limitation, learning-based MPC integrates machine learning techniques, particularly Reinforcement Learning (RL), that can enhance adaptability and closed-loop performance. This integration can result in more responsive and robust traffic management systems capable of handling uncertainties and dynamic environments.

The integration of MPC and RL has emerged as a compelling approach to control complex nonlinear systems, capitalizing on the strengths of both methodologies. RL constantly improves control policies based on observed system dynamics and performance feedback, while MPC automatically chooses the best control actions over a limited prediction horizon while still enforcing system constraints. This thesis investigates the integration of RL within the MPC framework for highway traffic flow optimization, specifically focusing on the dynamic regulation of Variable Speed Limits (VSL) and Ramp Metering (RM). The proposed approach is evaluated against three benchmark models: a theoretically optimal MPC model with perfect system knowledge, assuming exact parameters; an MPC controller with an imperfect prediction model, where these parameters deviate from their true values, leading to suboptimal control decisions; and the MPC-RL model, which dynamically adjusts the learnable parameters during training to improve overall system performance.

Deep Learning (DL) has transformed computer vision, leading to significant
progress in areas like autonomous vehicles and industrial automation. However, its application in underwater environments remains challenging due to factors such as light absorption, scattering, and water turbidity, which degrade image quality and hinder DL model performance. This thesis addresses these challenges by enhancing DL-based computer vision techniques for underwater scenarios, with a focus on autonomous robotic litter collection from the seabed. The work targets key limitations such as data scarcity, visual degradation, and scene variability, and proposes domain-informed approaches to enhance model generalization. The contributions cover the full DL pipeline, starting with the design of representative training data that supports object detection in shallow-water conditions, providing a benchmark for training and evaluation of detection algorithms. A multi-robot system is developed that integrates aerial, surface, and underwater vehicles to perform collaborative litter detection and collection. The thesis presents the system design, deployment, and the role of computer vision in the operational workflow. To address image degradation, an automated framework is proposed for selecting image enhancement methods based on task-specific performance metrics. Furthermore, environment-specific neural networks are introduced to handle variability in turbidity and lighting. Generalization to Out-of-Distribution (OOD) data is further addressed through a hybrid classification framework that combines a Convolutional Neural Network (CNN) with a physics-based classifier using the Moving Horizon Estimation (MHE) framework. Their outputs are fused via Dempster-Shafer theory to enable decision-making in unfamiliar scenarios. Finally, domain-informed neural networks are proposed to integrate physics-based knowledge into the DL pipeline via knowledge distillation. This method improves generalization and reduces dependence on large labeled datasets. The proposed methods are validated through simulation and real-world deployments, demonstrating improved performance and adaptability. Together, these contributions provide an integrated framework for deploying DL-based perception systems in challenging underwater environments.

Traffic Signal Control (TSC) in urban areas is typically performed by (adaptive) cycle-based methods. These methods are characterized by their robustness and simplicity, but do not hold the potential to achieve optimal control. Model Predictive Control (MPC) is a control method that has proven to be a great solution for urban TSC in literature. MPC is a model-based control method that predicts and optimizes future states over a moving prediction horizon. This approach holds the potential to foresee events that disrupt traffic networks, and actively adjust signal behavior to anticipate these disruptions.

This thesis research investigates an MPC approach for urban TSC, where a bridge opening occurs multiple times a day at a busy junction in Leiden, Netherlands. A linear prediction model is used with linear inequality constraints and binary control inputs. Multiple scenarios are simulated, where the MPC controller is compared to a baseline controller developed in the COCON optimization tool. The first scenario is in regular traffic conditions, so no bridge opening. In other simulations, a 5-minute bridge opening is incorporated. Finally, an upstream intersection is included to manipulate upstream traffic flow.

The simulation results show a decrease in average delay time per road user of 20% for normal traffic conditions. When the bridge opening was included, a delay decrease of 25% was achieved. When the upstream intersection was included, a delay decrease of 41% was achieved, using a centralized approach.

Over the years, the conventional open-loop basal-bolus regiment has proven to be inadequate for long-term glucose management of Type-1 diabetes mellitus (T1DM) patients. The ’artificial pancreas (AP)’, which is characterized by a closed-loop control system that typically relies solely on subcutaneous glucose measurements as a feedback to deliver insulin corrections, is widely regarded as a viable alternative to the former approach. An earlier literature survey showed that model predictive control (MPC) and proportional-integral-derivative (PID) control are the two most commonly employed approaches in APs to treat T1DM, especially the former in recent years, due to its ability to incorporate flexible safety-critical constraints and account for inherent system delays. With most state-of-the art MPC controllers designed to exert direct control on insulin corrections over the patient however, the extent to which closed-loop performance can be maximised greatly depends on how accurately the plant is modelled. In addition to the immense difficulty in identifying certain physical parameters and non-linearities that describe an individual patient’s insulin-glucose pharmacokinetics, practical challenges arising from the collection of open-loop data from the patient hinder the identification process of a prediction model. The resulting model-plant mismatch is detrimental to the closed-loop performance, especially during postprandial periods when hypoglycaemic episodes are most likely to occur. As a countermeasure for the model-plant mismatch, insulin delivery rates are often tightly enforced, which may increase robustness of the MPC controller but comes at a price of reduced, sub-optimal performance due to worse-case conservative assumptions.

This thesis presents a dual-layer control scheme: a lower layer comprising of a control architecture that regulates the rate of insulin delivery for corrections against disturbances, and an upper layer that represents a performance-driven tuning framework using Bayesian Optimisation (BO). The proposed control architecture is hierarchical, consisting of an inner-loop PID controller, and an MPC which serves as the outer-loop controller that governs the reference input signal to the inner-loop control system. The upper layer tuning framework is implemented in two stages, the first on a set of clinical-related parameters, and the second a controller and model parameter, as part of the individualized control objective. In both stages, the BO framework optimises the parameters via a daily query of one parameter set and its corresponding performance value, which is computed based on experimental data. The proposed control scheme is designed and evaluated using a cohort of 10 adult virtual patients of the U.S. Food & Drug Administration (FDA)-accepted University of Virginia/Padova simulator through several in silico scenarios. Numerical evaluation and results show that the proposed hierarchical controller outperforms its baseline counterpart, and that glycaemic performance of all the tested subjects improved using the proposed tuning approach compared to under nominal settings. Notwithstanding the absence of a probabilistic safety-critical constraint in the BO problem, the employment of a barrier-like penalty in the objective function, subjected to a deterministic domain-based constraint that is augmented using a rule-based approach, demonstrated to be an efficient safeguard against hypoglycaemia during the parameter tuning process.

This thesis explores a Bayesian Optimization technique for improving the tuning process of Model Predictive Control systems applied to soft robotics. Due to their high compliance and actuation redundancy, soft robotic systems are challenging to control through traditional rigid control frameworks. The objective of this study is to automate the hyperparameter tuning of MPC in order to enhance adaptability and efficiency in the control systems, handling the intricate behaviour of soft robots.

The study employs BO, leveraging its capability to efficiently navigate complex and high-dimensional optimization landscapes through a Gaussian Process (GP)-based surrogate model. This allows a systematic exploration and exploitation of the hyperparameter space toward setting up MPC optimal conditions that improve the performance metrics, such as response time and stability, under operational constraints. Two main acquisition functions, Expected Improvement (EI) and Lower Confidence Bound (LCB), are evaluated for their effectiveness in balancing exploration of the parameter space with exploitation of promising regions.

Obtained results, from simulations analyses, show that BO significantly reduces the manual effort involved in MPC tuning. The study also looks into the performance of the BO approach under varying initial conditions and introduces variance to the weights of the MPC to analyse the performance of BO under uncertainty.

This study addresses a critical aspect of automated driving: enhancing safety during highway emergency scenarios. This is achieved by formulating the problem within the framework of Stochastic Model Predictive Control (SMPC). In SMPC, safety constraints are designed such that a small chance of constraint violation is accepted, contrary to conventional Model Predictive Control (MPC), where hard constraints are imposed on the optimization problem that may never be violated. In the state-of-the-art, the exploration of SMPC for motion planning is minimal and focuses mostly on optimistic motion planning, depending on conservative backup solutions in emergencies. This research contributes to the state-of-the-art by designing a collision-avoiding, risk-averse, emergency motion planner by formulating the problem as SMPC, acknowledging the stochastic nature of real-world driving environments. By formulating the emergency planning problem specifically as SMPC, including risk minimization and providing a reference lane where the collision risk is minimal, the motion planner successfully navigates the automated vehicle through complex, dynamic emergency scenarios involving both static and dynamic obstacles without collision. In addition, the Interactive Multiple Model Algorithm (IMM) is used to estimate the maneuver intention of other vehicles (OVs). These maneuver estimates are used to formulate the probabilistic constraints in SMPC.

Furthermore, two novel backup controllers are designed to compute a fast, collision-avoiding trajectory when SMPC is infeasible. From the results, the backup controller based on MPC, designed such that it is always feasible and provides support in finding a feasible SMPC solution in the next iteration, showed the best performance.

The success of the proposed emergency motion planner is presented through simulation in 6 complex scenarios, where the planner safely navigates the automated vehicle through the emergencies without collision. The results are compared to related state-of-the-art, showing its effectiveness.

Urban rail transit networks are dedicated to providing safe, efficient, and eco-friendly transportation services for passengers. This thesis focuses on innovative model predictive control (MPC) strategies for the integration of passenger flows, timetables, and train speeds in urban rail transit networks. We introduce several innovative MPC frameworks, including bi-level MPC, scenario-based distributed MPC, learning-based MPC, and cooperative distributed MPC. These approaches exhibit significantly improved performance compared to conventional methods.

Greenhouses allow production of crops that would otherwise be impossible. Permitting more local, fresher and nutrient richer crop production. Eorts are taken to minimize societal harm due to energy and resource consumption by greenhouse production systems. One way to control such systems is by using model predictive control. Optimal crop yield and resource eciency can, in theory, be achieved by model predictive control. Unfortunately, one major drawback of model predictive control is that it is not well equipped to deal with parametric uncertainty. Significant prediction errors can occur when a mismatch between the model and the real system exists, resulting in deteriorated performance of the system. Strategies exist, such as robust MPC, that are designed to handle uncertainty, but those often result in conservative control policies. This thesis proposes to use model predictive control as a function approximator for RL in order to learn values for model and MPC parameters that can deliver optimal performance in the case of model mismatch.
In this thesis, data-driven economic nonlinear model predictive control using Q-learning is proposed as a method to alter the model parameters. The performance of the system af- ter learning is compared to approaches using robust and nominal model predictive control. Three dierent goals are determined: maximizing economic profit, minimizing the constraint violations and maximizing the economic performance while minimizing constraint violations.
In this work, an ENMPC scheme is used as a function approximator in a Q-learning envi- ronment. The optimization solution from the ENMPC scheme is used as the input to the system, while the Q-learning agent optimizes the parameter values of the ENMPC scheme and model for the environment. The performance of the system after learning is compared to approaches using robust and nominal model predictive control. The simulation results show that the data-driven ENMPC using reinforcement learning is able to decrease constraint vi- olations by up to 94%, but unable to increase economic performance compared to nominal MPC, compared to robust MPC the EPI is increased by almost 10% while keeping constraint violations at a similar level.

Nowadays, the high demand for road transportation often reaches a point where it exceeds the capacity of freeway traffic networks, resulting in congestion. Freeway traffic congestion is a major social problem, as it is the reason for increased time delays, higher accident risk and environmental pollution. There is, therefore, the need for a sustainable solution that can be implemented on the existing road infrastructure. Freeway traffic control is considered as such a solution. It uses different control measures, in order to improve the performance of the freeway traffic network, by influencing the drivers’ behaviour.

The Variable Speed Limits (VSLs) are a traffic control measure that aims to increase traffic safety, improve traffic flow and reduce the environmental pollution. Towards the improvement of the freeway traffic flow, easy-to-implement VSL control algorithms are used as mainline metering control approach.

Two easy-to-implement VSL algorithms are reported, namely the Mainstream Traffic Flow Control (MTFC) and the Logic-Based control algorithm for Variable Speed Limits (LB-VSL). The algorithms are usually implemented in an non-adaptive framework. The main contribution of this thesis is that it proposes an adaptive framework for both algorithms, in which the critical density of the freeway traffic network at bottleneck’s location is estimated on-line. This estimated critical density is used to adjust the controllers’ parameters in real-time.

Three different estimation methods for the bottleneck’s critical density are studied, namely the Parameter Estimation (Parameterschatter) method, the Simple Derivative Estimation (SDE) and the Kalman-Filter-based derivative Estimation (KFE) methods. All three methods focus on the real-time estimation of the derivative of the Fundamental Diagram (FD), in order to produce estimations of the critical density.

A case study is performed to evaluate the performance of the three algorithms. A hypothetical 12 km long freeway stretch is used, which contains two VSLs installations and a lane-drop bottleneck. In the first part of the case study an accident scenario is studied, which decreases the critical density at the bottleneck’s location. The second part of the case study evaluates the three adaptive easy-to-implement VSL algorithms under the assumption of a decrease of the critical density across the whole network, due to bad weather conditions.

The global market for personal mobility has transformed over the last decade. Traditional taxi services have to compete with the emergence of ride-hailing services such as Uber and Lyft. Rapid developments in available algorithms and real-time inter-connectivity of travellers and vehicles offer mobility-on-demand (MOD) services new possibilities to maximise the efficiency of the ride-hailing process. It is well established that rebalancing can enable the true potential of a ride-hailing fleet. In this context, rebalancing is defined as the redistribution of idle vehicles over the service area of a ride-hailing operator. In the search for the optimal rebalancing strategy, model predictive control (MPC) and approximate dynamic programming (ADP) methodologies are active fields of research. However, the computational burden of MPC limits the length of the predictive horizon in large MOD applications. This thesis proposes a novel algorithm that combines ADP and MPC. More specifically, we integrated a value function into the MPC framework as a terminal cost. We shorten the horizon of MPC and propose to use the value function as a long-term planner. For the terminal cost, we continue research into piece-wise linear value function approximation. We implement a novel multi-period approximation of the value function, where we use the maximum repositioning length as the horizon. In our case study, the value-based repositioning strategy offers comparable service quality to conventional MPC at 5.2% of the computational burden. The hybrid ADP-MPC algorithm offers flexible horizon partitions between the multi-period value function and MPC. It addresses the shortcomings of both ADP and MPC algorithms in repositioning problems. At peak performance, it offers an increase of 4.5% in service rate and a decrease of 5.2% in the average waiting time over conventional MPC, at roughly half the computational burden. Keywords: Ride-hailing, Mobility on demand, Model predictive control, Approximate Dynamic programming

Speed trajectories considerably influence vehicular fuel consumption, particularly on signalized roads. To minimize fuel consumption, sharp acceleration/deceleration maneuvers and idling events at signalized intersections should be prevented. By taking advantage of the technological developments in infrastructure-to-vehicle communication, the possibility of receiving traffic signal phase and timing information in advance is enabled. Although a vast amount of research has been dedicated to optimal speed trajectory planning, existing methods may not be adequate in identifying the optimal solution for vehicles driving on signalized roads. Most studies do not involve queue estimation in the algorithm, which makes it challenging to deploy these methods in practice. Moreover, research efforts focus on undersaturated traffic conditions where queues can completely dissolve in a single cycle. Once the network is oversaturated, residual queues are formed generating traffic fluctuations and complete stops, significantly reducing the effectiveness of the application.

In this thesis, an optimal control problem is formulated to obtain the optimal speed trajectory, where traffic induced constraints are taken into account and queue estimation is explicitly integrated into the control framework. Based on kinematic wave theory, an efficient and accurate procedure to formulate the queue constraints in various traffic conditions is developed. To facilitate real-time control actions, the constrained optimization problem is solved using model predictive control. The simulation case studies show the proposed algorithm achieves vehicular fuel consumption savings as high as 29.15% compared to an existing approach in the literature. However, the fuel consumption savings are at the expense of an increase in travel time up to 1.65% compared to the literature approach. The results also indicate the benefits grow with increasing market penetration rates (MPRs) of controlled vehicles until it levels off at about 80% MPR. Furthermore, the results demonstrate the proposed algorithm can deal with stochasticity in traffic behavior. Finally, the thesis highlights the need for future research to further improve the proposed algorithm.

Cycling is an increasingly attractive transportation mode, thanks to its health and environmental benefits. Personalized travel assistance services can help make cycling more appealing by providing speed or route advices that can reduce travel time and increase safety while taking into account the personal preferences of cyclists. Due to its ability to learn agents' reward function, Inverse Reinforcement Learning is a suitable algorithm for learning cycling preferences from data.
This thesis aims to describe cycling styles as a set of cycling preferences encoded as a reward function composed of a weighted sum of features. The weights associated to the features composing the reward function represent the importance given to each cycling preference and express the trade-off between different goals of a cyclist. Continuous-time Inverse Reinforcement Learning extracts the weights from empirical cyclists' trajectories collected during an experiment performed in Delft. During the experiment, cyclists were asked to cycle according to three different cycling styles: cautious, normal and aggressive. Differences between weight sets extracted for each cycling styles were analyzed by means of the Kruskar-Wallis statistical test and K-Means clustering algorithm, and the averaged weights for each cycling style were used to simulate a set of test trajectories.
It is shown by simulations that the reward function identified for a specific cycling style leads to an improvement in terms of similarity to test trajectories with the same cycling style with respect to the reward functions corresponding to other cycling styles. The statistical analysis shows that the weights of cautious and aggressive cycling styles show statistical differences and define separate clusters.

Stochastic Dynamic Programming (SDP) has shown promising results for sequential decision problems of the route optimisation for an Electric Vehicle (EV) with the presence of stochastic variables in the travel cost. However, in studies, the optimisation problem formulation for EVs has been lacking in detail. For example, possible waiting times at a Charging Station (CS) have been neglected. This thesis uses SDP to formulate a more holistic optimisation problem for EVs moving through a road network where travel speeds and charging station availability are stochastic. The goal is to optimise the travel costs, which consists of, e.g., the journey time and the charging cost, for an EV on long-haul trips.

In this thesis, four simulation-based case studies are conducted: (1) comparison of conventional navigation system with the proposed method; (2) speed optimisation in order to improve the travel costs; (3) charging platform selection in order to improve the travel cost; (4) uncertainty influence on the travel costs. The case studies are conducted to create insight into how the travel costs of an EV can be optimised. In these case studies, the influence of multiple factors has been taken into account and investigated. For example, cabin climate control, which is dependent on the ambient temperature, has a significant influence on the energy consumption of the EV resulting in higher travel costs.

The simulation results have shown interesting results. Compared to a Min algorithm, which uses a strategy to minimise the travel and charging time, the proposed method can find an optimal policy that is in some cases 5% shorter in terms of journey time. It is profitable for certain ambient temperatures and maximum allowable driving speeds in terms of journey time and charging cost to optimise the driving speed below the maximum allowed driving speed on highways. This results in a shorter journey time and saving charging costs. For example, for a maximum speed of 120 (km/h) and an ambient temperature of 20 ◦C, 3% of journey time advantage can be achieved by optimising the driving speed.

Research has shown that GLOSA systems can help reduce the emission of $CO_2$ by motor vehicles and improve flow on urban road networks. However, existing GLOSA systems only work in combination with a limited selection of Traffic Signal Controllers (TSCs) and therefore have not been widely implemented. This research describes and evaluates the design of a system to model the behaviour of the most common types of TSCs using data that is shared by road users, also known as Floating Car Data (FCD). Almost every road user is able to share his location using a smartphone, so this means that a system of this kind can be implemented everywhere in the world where people are willing to participate, without requiring any changes to the infrastructure. It is chosen to model TSC behaviour using Hidden Markov Models (HMMs), because they can be generated from unlabelled data, as opposed to other machine learning techniques. More specifically, Explicit Duration Hidden semi-Markov Models (EDHSMMs) are used, so that timing behaviour can be captured more accurately than with standard HMMS. An adaptation to an existing learning algorithm is made to decrease amount of data required for learning and to make sure the algorithm can cope with the sparse nature of FCD. Additionally, a system is developed to combine generally known intersection data and FCD to generate models that have transition parameters with low entropy and thus high predictive powers. In this way, the system produces models that are accurate enough to be used for the generation of GLOSA from relatively sparse data.
In a simulated setting, it is shown that the system can identify the true switching behaviour of the most common types of TSCs and that it produces models whose duration probability distributions converge to the true situation when provided with realistic amounts of data.

In decision making problems, the ability to compute the optimal solution can pose a serious challenge. Dynamic Programming (DP) aims to provide a framework to deal with a category of such problems, namely ones that involve sequential decision making. By dividing the original control problem into sub-problems and solving it backwards in time, from the end of the time horizon to the start, the method can compute a map of optimal solutions with respect to the initial condition. In order to divide the original problem into subproblems the DP method takes advantage of the principle of optimality, which states that a sub-solution of the optimal solution should be the optimal solution for the equivalent subproblem. In control systems, where the state and decision spaces are continuous, the original DP framework can be intractable due to the size of the discretization needed to simulate the continuous space. Therefore, efficient approximations are needed to solve such problems. One promising method is called Conjugate Dynamic Programming (CDP). The CDP algorithm is able to transform the original framework and solve problems in the conjugate domain providing a computational advantage over the standard method. In this work, we aim to improve and extend the setting under which the CDP algorithm operates, thus providing a more concrete advantage over standard method . In that regard, we will extract the optimal control actions from within the CDP algorithm, eliminating the need for solving an extra optimization problem for their computation. In addition, we will introduce a different interpolation technique that can outperform the current one, in certain scenarios, thus granting the user more choice when solving a decision making problem.

Being a safe and healthy alternative for polluting and space-inefficient motorised vehicles, cycling can strongly improve living conditions in urban areas. Idling in front of traffic lights is seen as one of the major inconveniences of commuting by bicycle. By giving personalised speed advice, the probability of catching a green light can be increased whilst taking the cyclist preferences into account. Due to its adaptive properties, Reinforcement learning (\acs{RL}) is a suited algorithm for developing optimal speed advice policies when dealing with a dynamic traffic environment and unique cyclist preferences. Generally, a large amount of training samples is required to successfully train a \acs{RL} algorithm. This poses a problem for this specific application since training samples must be generated by humans and are therefore scarce. Moreover, exploration of the environment is challenging since humans will not comply with irrational speed advice. These factors currently restrain the practical implementation of \acs{RL} algorithms for giving speed advice. This thesis aims to overcome these problems whilst maintaining a competitive performance compared to conventional \acs{RL} algorithms. This is done by using function approximators and a combined planning and learning method called Dyna. During a case study, three different function approximators are compared to reduce the amount of required training samples, namely polynomial functions, radial basis functions, and artificial neural networks. Secondly, the effectiveness of Dyna to improve the quality of the speed advice in an unknown environment is assessed. Finally, these methods are applied in a framework focused on the practical implementation of \acs{RL} for giving speed advice. It was concluded that function approximation method can significantly reduce the amount of required training samples to train a \acs{RL} algorithm. Dyna can increase user retention by providing cyclists with a high quality speed advice algorithm during the early learning phase of the algorithm. Therefore, it can be concluded that this \acs{RL} approach for giving personalised speed advice to cyclist approaching intersections is practically implementable and can even outperform benchmark algorithms in terms of travel time, energy consumption, and safety.

Advancements in knowledge and technology have led to an evolution of greenhouse operations: from simple transparent shelters to (one of) the most profitable sectors in agricultural industry. The understanding of the physiological and biological processes of the inside micro-climate and crop, as well as the introduction of climate control, were critical for improving both the quality and production rate. Advanced control strategies like MPC involve the application of horticultural and plant physiological knowledge using mathematical models that describe the dynamical behavior of the greenhouse and crop. An on-line control method balances the benefits of crop yield and costs of the climate control equipment. The increasing production and cultivation rate in response to the growing demand is at a point where the focus and tension is shifted to the energy input and environmental footprint of greenhouse operations. This stimulates the agricultural industry to provide sustainable solutions. An innovative greenhouse is developed by BBBLS Solutions, that uses a soap bubble cavity which improves the thermal properties and energy efficiency. The concept is in development phase and is acknowledged that there is potential for improved climate control and automation of the control of the soap bubble cavity. This research introduces a MPC strategy for controlling the greenhouse climate and the soap bubble cavity. The greenhouse processes are described by a mechanistic model, which simulates the inside climate variables as a function of the greenhouse structure, outside environmental conditions and climate control equipment. The dynamics of the soap bubble cavity and the corresponding thermal varying properties were then augmented in the climate model. The augmented model is used as a predictor for the future dynamics of the greenhouse climate, for which the optimizer generates a control input sequence for the climate control equipment and soap bubble generators inside the cavity. The MPC implementation results demonstrate that the greenhouse temperature can be controlled within the pre-defined bounds and that the filling of the soap bubble cavity is with respect to outside conditions and priorities. Furthermore, the MPC strategy supports real-time control as the CPU time can be less than the time between control instances. The proposed methodology is an initial attempt but proves to be a promising concept for further development.