The growing penetration of renewable energy sources (RES) in power networks introduces significant challenges in load frequency control (LFC). Uncertainties in power generation make load balancing difficult, leading to frequency fluctuations that can cause equipment damage and blackouts. Additionally, the large-scale, spatially distributed nature of modern power systems necessitates a multi-agent control approach. Traditional PID-based controllers are ill-equipped to handle the uncertainties introduced by RES, while stochastic and robust model predictive control (MPC) methods, though capable of addressing small uncertainties, are often overly conservative. Similarly, reinforcement learning (RL) offers adaptability but lacks interpretability and explicit constraint handling. This thesis presents a distributed control framework that integrates model predictive control and reinforcement learning to address these challenges. Parametric uncertainties are incorporated into the system dynamics to account for stochasticities introduced by RES. At the core of the approach is a parameterized MPC scheme that approximates the RL value function, enabling the system to learn to avoid constraint violations while optimizing performance by driving state deviations from nominal operating conditions to zero. A distributed Q-learning scheme is used to learn the parametrization, which reduces the need for extensive information sharing, enhancing cybersecurity, and enables learning even with imperfect initial knowledge of system dynamics. The proposed framework is applied in simulations of a three-area power network to evaluate its potential and is compared against stochastic MPC and a deep deterministic policy gradient learning method. Results show that the proposed approach offers a balance between adaptability, performance and interpretability, and successfully handles constraints. It outperforms sample-based stochastic MPC in terms of cost, computation time and constraint handling, and outperforms deep deterministic policy gradient RL in performance, constraint handling and sample efficiency.

This research investigates the application of different deep reinforcement learning methods for optimizing traffic light control in multi-modal urban traffic environments using the SUMO traffic simulator. Urban traffic congestion, with its significant economic, environmental, and social impacts, necessitates more sophisticated control strategies that can adapt to varying traffic conditions. Traditional traffic control systems, like fixed-time and adaptive methods, are often insufficient in handling the complexity of multi-modal traffic, which includes various traffic modes such as passenger cars and buses. Deep reinforcement learning, with its ability to dynamically optimize traffic light control without requiring prior knowledge of traffic patterns, is a promising method to improve traffic efficiency and achieve transit priority in multi-modal traffic. The research aims to address the limitations of the existing relevant research by employing several deep reinforcement learning algorithms, particularly multi-agent deep reinforcement learning methods, to coordinate multiple traffic lights in SUMO simulation. Research experiments are conducted in three different cases, which are set in road networks of different sizes respectively, and fixed traffic light control and max-pressure traffic light control are implemented for comparison. The applied deep reinforcement methods are evaluated in terms of training process and model evaluation. And the research results demonstrate that deep reinforcement learning methods, especially multi-agent deep reinforcement learning methods, can significantly enhance traffic flow efficiency and achieve transit priority in complex urban settings in multi modal simulation.

Timetables determine the service quality for passengers and the energy consumption of trains in metro systems. In metro networks, a timetable can be made by optimizing train departure frequencies for different periods of the day. Typically, the optimization problem that arises from optimizing train departure frequencies in metro networks involves integer variables, which can cause the problem to be computationally too complex for real-time applications. The main objective of this thesis is to reduce the computational complexity of optimizing train departure frequencies in metro networks while maintaining a relatively accurate solution.

In this thesis, we first apply classical Benders decomposition to optimize train departure frequencies in a metro network considering time-varying passenger demands. Subsequently, we apply an $\epsilon$-optimal Benders decomposition approach to reduce the computational complexity further. A simulation-based case study using a grid metro network illustrates the performance of the two Benders decomposition-based approaches.

The simulation results show that the classical Benders decomposition approach significantly reduces the computational burden of optimizing train departure frequencies in metro networks. Moreover, the $\epsilon$-optimal Benders decomposition approach can further reduce the computation time when the problem size increases of the optimization problem when compared to the classical Benders decomposition approach while maintaining an acceptable level of performance.

This thesis extensively examines the influential factors affecting the performance of approximations of Model Predictive Control (MPC) control laws using neural networks. MPC is a control strategy that solves an optimization problem at each timestep. This problem can be computationally complex and could be too slow to compute for online control. Sometimes an explicit solution for MPC exists, but this can become very large in memory and is not always available. That is why approximations with neural networks might offer a benefit. Under certain conditions, the explicit solution yields a piecewise affine (PWA) control law. A PWA model class is equivalent to the so-called Max-Min-Plus-Scaling (MMPS) model class, which is a generalization of max-plus and min-plus algebra. Neural networks are made up of neurons, which make use of activation functions. A feed-forward neural network with some specific activation function can yield an MMPS function. This inspires us to research the use of different activation functions in approximating MPC control laws. Additionally, we investigate different sampling strategies and the use of max-plus and min-plus layers in neural networks.

We do this by setting up different PWA and non-PWA control laws for two inverted pendulum systems and training several neural networks to approximate these control laws. We first observe a significantly better performance in approximating the PWA control laws compared to the non-PWA control laws. When varying the activation functions of the neural networks we find that for PWA control laws a MMPS activation function can offer a better performance, but it is not guaranteed for all MMPS functions. We also find that networks with custom max-plus layers can offer a similar performance on approximating control laws compared to networks with traditional layers. When investigating what sampling strategy is most beneficial we find comparable performance with a stratified sampling strategy and a uniform sampling strategy. Depending on what areas of the control law you want to capture with the most detail, you can choose the most viable sampling strategy. With this, we have researched various factors that influence the performance of approximations of MPC control laws. The thesis ends with a recommendation to research even more factors that might offer even better approximations.

Research in passive Heating, Ventilation, and Air Conditioning (HVAC) systems has gained traction over the last few years. Although passive HVAC is not a new concept, advances in environment sensing, control methods, and hardware have made it a more viable method. Some difficulties still exist, such as optimal sensor placement and optimal control strategies. Sensor selection is an important aspect of HVAC design. The system can become difficult to control with incorrect placement of sensors, resulting in higher energy consumption, lower comfort levels, or poor air quality.
There are essentially three methods to determine the optimal sensor location: model-driven, data-driven, and simulation-driven. The model-driven methods use mathematical models to maximize the observability of the system but are mostly used for simplified simulated rooms. Data-driven methods often use clustering algorithms, or maximize metrics such as entropy or mutual information. These methods focus on estimating the indoor air temperature distribution. Simulation-driven methods use simulations to determine the airflow or temperature
fields, often with CFD. These are used to find local hot spots or locations for fast detection of contaminants. No research was found that used sensor data of additional building components besides of the indoor air temperature.
In this work, the sensors are selected based on model prediction accuracy and the overall control performance to determine the effect of addition state measurements. A model is constructed to simulate the building, together with an MPC and an extended Kalman filter for state estimation. These are combined to run the optimization and determine the control performance. The sensor set average of each measured state is considered the true temperature. For all possible sensor combinations, the error of the combination average w.r.t. the true temperature is assumed Gaussian. The fitted Gaussian error distributions are then used as measurement noise in the model. The building and control response is simulated
with the measurement error over multiple days. Two algorithms are implemented to find the optimal sensor set: a predictive method and greedy method. The results are compared to each other and both methods showed that the indoor air temperature measurements have the largest effect on performance. Measuring additional states only resulted in a small increase in performance.

Both model predictive control (MPC) and reinforcement learning (RL) have shown promising results in the control of traffic signals in urban traffic networks. There are, however, a few drawbacks. MPC controllers are not adaptive and therefore perform suboptimal in the presence of the uncertainties that always occur in urban traffic systems. Although very advanced prediction models for urban traffic signal control systems exist, these models also come with a price: the computational complexity of MPC controllers increases with the accuracy of the model. RL techniques involve a time-consuming and data-dependent offline computation, as the agent needs to pursue a training process. The training process is also the main reason why RL techniques have not been employed in real-world urban traffic systems. Through exploration in the training phase the controller may cause a suboptimal and potentially unacceptable bad performance in the system. Besides, most RL techniques do not have any stability and feasibility guarantees. With the goal of mitigating these drawbacks, the model-reference RL adaptive control framework is introduced. RL is used to obtain an adaptive law to adjust a stable baseline controller to follow a set reference. This thesis focusses on the design and analysis of this scheme where MPC control is used to obtain the baseline control input. The computed baseline control input combined with the traffic model used, determines the reference state to be followed. By performing a case study, the training characteristics of the framework are compared to those of a conventional RL-based controller. Besides, the system performance framework is compared to that of a fixed-time controller a conventional MPC controller and a conventional RL-based controller. The simulation shows that the framework outperforms the RL-based controller in terms of performance during training and the general simulation performance of the MPC controller.

There is an urgent need for technical innovations in the construction industry to meet the European Parliament’s qualifications for reducing the energy footprint of buildings. This thesis is devoted to conducting research on control strategies that optimally manage passive energy sources, e.g., natural ventilation and solar irradiance, in combination with an active energy source. The main objective is to develop a control strategy that is both on-line applicable and maximizes the performance of the building energy management system in terms of passive fraction of energy while maintaining indoor thermal comfort. Energy-saving model predictive control (MPC) structures have been extensively researched in the literature on building energy management systems. These studies usually focus on the energy distribution in multi-zoned buildings and rarely consider optimal control of a single or multiple passive energy sources. In addition, most studied building models are based on general buildings and are established by means of simulation software tools. A more experimental study has not yet been conducted on optimal energy management systems for a building that is maximized in harvesting passive energy. This work investigates five MPC strategies as a way to optimize the operations of four solar blinds, a thermal chimney, and an active energy source. In these strategies, linear and nonlinear MPC are considered in the forms of centralized and hierarchical architectures. White-box modeling and linearization methods are adopted to develop the required linear and nonlinear building models. Thereafter, the proposed modeling methodology is validated by using experimental data. The hierarchical MPC architecture that considers a hybrid structure with a linear MPC agent for solar blind operations, a nonlinear MPC tracker for ventilation, a linear Kalman filter, and separated state-update loops appears to be the best-performing strategy. This control structure is also applied in a case study, in which it is tested on experimental data from the real-case office building, which is controlled by a rule-based control structure. The results show that the developed control structure is able to outperform the rule-based controller in terms of minimizing energy consumption and maintaining thermal comfort.

2getthere specialises in autonomous people transport through their GRT vehicle, used for transporting people at the airport from the parking to the terminal to ensure the GRT can operate comfortably and safely in a mixed traffic environment. The vehicle needs to plan a smooth and collision-free path. An essential aspect of a safe path is to predict the motion of the surrounding vehicles of the GRT to guarantee a smooth adaption to the constantly changing traffic scenario. The smooth adaption to the changing traffic scenario is essential for the GRT. The GRT cannot brake with the same magnitude as surrounding traffic due to standing people inside the vehicle. The importance of the smooth adaption to the traffic scenario resulted in the following problem statement; is it possible to create a motion prediction model for the surrounding vehicles of the ego vehicle to allow for pro-active velocity planning of the ego vehicle itself. From the literature, motion prediction models can be divided into physics-based motion models, manoeuvre-based motion models, and interaction-aware motion models. Each level has a certain amount of building blocks, with each level being an extra addition of situation awareness to the previous level.
An interaction-aware motion model is the most advanced prediction model taking the infrastructure and interaction between the vehicles in the traffic scene into account. Therefore this principle of motion prediction is chosen for the motion prediction of the surrounding vehicles of the GRT. First, all possible state and route information of the surrounding vehicles are gathered. Secondly, a tree is created based on the vehicles' routes in the traffic situation, with the unique combination of manoeuvres known as a single branch of the scenario tree. Hereafter, the possible conflict areas between the individual manoeuvres are determined based on the desired trajectories for a specific route. The first vehicle passing the conflict area will influence the second vehicle's trajectory for passing the same conflict area, which will influence the third vehicle's trajectory and so on. Each vehicle can avoid a collision by passing in front or behind the previous vehicle(s) at the conflict area. Evaluating all passing possibilities will result in the velocity trajectory with the lowest cost, constrained by comfort. The sum of all costs for each branch of manoeuvres is used as a metric to determine the possible velocity profiles for each vehicle in the current traffic situation, resulting in a better understanding of the traffic scenario and ensuring a comfortable adaption to a changing traffic scenario. This thesis will evaluate several methods found in the literature to built a motion prediction model suitable for the application at 2getthere. The chosen motion model will be evaluated by two scenarios in a T-junction intersection.

Over the past few decades, transport authorities globally have resorted to transport models for testing policy interventions and simulating the results as part of the ex-ante analysis. Within the domains of traffic assignment, there is a greater focus on the dynamic representation of traffic, which has proved to be more accurate when compared to their static counterparts. This has put Dynamic Traffic Assignment (DTA) Models at the forefront of development. Departing from the classical traffic flow theories Macroscopic DTA’s simulates aggregated traffic analogous to the flow of fluids or gases. This aggregation enables high-speed computation with the ability to achieve a stable equilibrium state within feasible model run times. Due to the large number of Macroscopic DTA models developed worldwide, the model user is posed with the problem of using the correct model for the correct application. The current research aims to provide an answer to this problem through the design, development, and validation of an evaluation framework for Macroscopic DTA’s. The objective evaluation of the DTA’s is performed through certain Measures of Performances (MoPs). The subjective side of evaluation showcases the differences in importance associated with model features which vary from model users to application domains. Three macroscopic DTA models popular in the Netherlands are used for the application of the framework: the MARPLE (Model for Assignment and Regional Policy Evaluation), StreamLine: MaDAM (Macroscopic Dynamic Assignment Model), and StreamLine: eGLTM (event-based Generalized Link Transmission Model). From the results, it is observed that For a Strategic Planning application, both MARPLE and StreamLine: eGLTM proved to be better alternatives, as they performed exceedingly better in achieving a stable state of convergence. However, as the time horizons of application became smaller as is the case with Tactical and Operational planning, the final score for StreamLine: MaDAM improved substantially due to its accuracy involved in link-level propagation and queuing. The evaluation scores also showcase the fundamental trade-off between model complexity and computational speed was visible from the results. We can observe variations across model users, which validates our original hypothesis that the right choice of a model primary depends on the person using it and the application it is deployed for.

Significant work has been done in the field of computer vision focusing on learning and clustering methods. The use of improved learning methods has paved a way forward for researches to explore various theories to improve existing methods. One among various learning methods is Hierarchical learning which has showed impressive benefits and performance over traditional sequential learning approaches. In general, machine learning models require a lot of data for every new scenario which is not always possible and if so, is very expensive. Transfer learning, which focuses on transferring knowledge across trained machine learning models, is a promising machine learning methodology for solving the above problem. In this thesis, we propose an end-to-end neural network architecture on the NM500 neuromorphic chip using an incremental hierarchical learning approach. We first design a hierarchical representation of a taxonomy, develop a batch of pre-classifiers and use their output to construct a custom feature vector that is the input to the front-end network which learns the taxonomy. In other words, the taxonomy is embedded in the clustering method and not trained by a backpropagation algorithm. The custom feature vector has been structured to accurately incorporate the taxonomy based on the Manhattan distance norm. The structure has been proven mathematically and validated using experiments. A Radial Basis Function (RBF) is used for learning and a combination of RBF and K-Nearest Neighbors (KNN) for classification. The applicability of the proposed framework has been demonstrated on a road sign classification problem which is represented as a taxonomy. The ability of the framework to incrementally learn new categories and update the taxonomy online has also been shown. Lastly, we show a case of transfer learning where the entire back- end networks is used as a starting point to learn new features without significantly forgetting prior knowledge. This transfer learning framework showed comparable performance to the standard learning method in terms of accuracy while using significantly less labelled data. This work paves a way forward for researchers to develop transfer learning frameworks and more importantly explore neuromorphic hardware for machine learning tasks.

Being a safe and healthy alternative for polluting and space-inefficient motorised vehicles, cycling can strongly improve living conditions in urban areas. Idling in front of traffic lights is seen as one of the major inconveniences of commuting by bicycle. By giving personalised speed advice, the probability of catching a green light can be increased whilst taking the cyclist preferences into account. Due to its adaptive properties, Reinforcement learning (\acs{RL}) is a suited algorithm for developing optimal speed advice policies when dealing with a dynamic traffic environment and unique cyclist preferences. Generally, a large amount of training samples is required to successfully train a \acs{RL} algorithm. This poses a problem for this specific application since training samples must be generated by humans and are therefore scarce. Moreover, exploration of the environment is challenging since humans will not comply with irrational speed advice. These factors currently restrain the practical implementation of \acs{RL} algorithms for giving speed advice. This thesis aims to overcome these problems whilst maintaining a competitive performance compared to conventional \acs{RL} algorithms. This is done by using function approximators and a combined planning and learning method called Dyna. During a case study, three different function approximators are compared to reduce the amount of required training samples, namely polynomial functions, radial basis functions, and artificial neural networks. Secondly, the effectiveness of Dyna to improve the quality of the speed advice in an unknown environment is assessed. Finally, these methods are applied in a framework focused on the practical implementation of \acs{RL} for giving speed advice. It was concluded that function approximation method can significantly reduce the amount of required training samples to train a \acs{RL} algorithm. Dyna can increase user retention by providing cyclists with a high quality speed advice algorithm during the early learning phase of the algorithm. Therefore, it can be concluded that this \acs{RL} approach for giving personalised speed advice to cyclist approaching intersections is practically implementable and can even outperform benchmark algorithms in terms of travel time, energy consumption, and safety.

Graph Neural Networks are a unique type of Deep Learning models that have a capability to exploit an explicitly stated structure of data representation. By design they carry a strong relational inductive bias, which is a set of assumptions that makes the algorithm prioritize some solutions over another, independent of observed data. This makes the method especially interesting for applications to problems, that are naturally relation-centric, or in which local interactions between features are the main value of interest. The presented research case, aims to explore GNN potential in application to an Ocean Acoustics problem. Using the geometric ray-tracing algorithm, BELLHOP, a large number of underwater sound propagation scenarios was simulated. Each scenario is described by a limited set of parameters and a Sound Speed Profile function. The latter, acting as a guideline for estimating paths of rays travelling through a water column, has a critical impact on sound propagation mode. For the data-driven model to effectively capture the acoustic phenomena, requires a mean of representing interactions in very scarce feature space and especially with respect to the nonlinear function representation of the sound speed. First, the solution of the problem is approached with a traditional Machine Learning model, a decision-tree algorithm XGBoost. In effect, some important characteristics of the collected data sample are revealed. Moreover, by testing inference capacity of the database with a reliable algorithm, gives an estimate of the properties of the Sound Speed Profile that have the biggest impact on sound propagation. It is proven, that with carefully engineered features, that include a degree of added expert knowledge, a standard model can achieve good accuracy of prediction. Secondly, a Knowledge Graph is designed to represent the whole context of explicitly stated expert knowledge, using concepts from Hydroacoustics. They are encoded in a form of relational structure connecting actual features of the data into logical categories. In this representation it can be used by the Knowledge Graph Convolutional Network model designed for the problem. A range of tests performed on KGCN proves that using a Graph Neural Network can be feasible to solve the problem, however it also reveals a range of issues regarding model's capability to handle the complexity of problem statement.

Reset control is a "simple" nonlinear control strategy that has the potential of being widely adopted and improving the performance of systems traditionally controlled with PIDs. Lack of suitable methods for proving stability, that are in line with the current industrial practice, hampers the wider acceptance of reset control. In this thesis, novel sufficient conditions for stability of reset control systems, that can be evaluated using measured frequency response function of a system to be controlled, are derived using the hybrid passivity and finite-gain framework. A method for analysing the hybrid passivity and finite-gain parameters of reset systems, that can be extended to other classes of nonlinear systems, is developed. Additionally, a variant of the “Constant in Gain Lead in Phase” reset element, that facilitates the use of the proposed method for the stability analysis, is introduced. Stability of several precision positioning systems with reset controllers, designed for different objectives, is studied to demonstrate the applicability of the proposed hybrid passivity and finite-gain approach for the stability analysis of reset control systems. Guidelines for design of reset systems such that their stability can be concluded using the hybrid passivity and finite gain method are shown. This thesis presents a new view on the stability of reset systems and addresses the need for frequency-domain tools for stability analysis of nonlinear control systems in precision mechatronics applications.

This thesis describes the application of a stochastic active fault diagnosis method for localising leaks in a Water Distribution Network (WDN) under parametric uncertainty, in order to localise leaks faster and more reliable compared to non-invasive state-of-the-art methods. Output residual Probability Distribution Functions (PDFs) for a discrete set of leak hypotheses are constructed by smoothing out the realisations resulting from Monte Carlo simulations of a non-linear hydraulic model by means of Kernel Density Estimation (KDE). Active pressure control inputs are designed to minimise the probability of misisolation aka the Bayes error. Where Pressure Reducing Valves (PRVs) usually regulate pressure at a minimum level, we show that they can also be used to enhance leakage diagnosis. During night time, when user demands are low, the control inputs are iteratively updated according to some objective function that aims to maximise the sum of stochastic distances between residual PDFs, weighed with their corresponding likelihood, whereas the likelihood vector itself is updated in a Bayesian classification framework. Stochastic distances are calculated using a stochastic metric that quantifies the overlap between residual PDFs. The algorithm is applied to the Hanoi benchmark network for different intensities of parametric uncertainty. Improvements in performance are observed in comparison to a passive cutting-edge counterpart method.

Substantial efforts are being made to make robots more reliable and safe to work around humans. Robots often perform flawless demos in a controlled environment under the supervision of an operator but tend to fail in the real world when deployed for a long period of time due to faults and environmental disturbances. A robotic system is composed of different physical and software components whose characteristics are likely to change over time. Assumptions made about the system during the design phase may change over time, especially when a system is deployed for long periods. Such changes that are often ignored, need to be considered. Environments in which a robot operates are dynamic with high uncertainty and unpredictability. In such scenarios, capabilities such as situational awareness and self-adaptation will be useful to create more robust, resilient and reliable solutions. The objective for this thesis work is to develop a framework which will embed capabilities such as situational-awareness, context-awareness and self-adaptation within a robot. This research provides a novel, reusable and generalised localisation framework called Situation-Aware Self-Adaptive (SASA) localisation framework for robotics application. This framework is developed using knowledge representation and reasoning which will provide a robot with the capability of adapting according to the situation. We have demonstrated the applicability of the SASA framework to a mobile robot localisation use case. In this research work, we have demonstrated the performance of the framework during environmental disturbances due to poor illumination and featureless environment and internal fault due to component failure. We have also demonstrated the reusability, changeability and the consistency of SASA framework. This work showed that the situational-awareness and self-adaptation capability enhances the robot’s localisation ability and provides reliable localisation even in the case of environmental uncertainties and internal faults where conventional localisation systems fail. This thesis represents a leap forward in the direction of creating more reliable and resilient solutions for robotic applications and it lays the foundations for further research in this direction.

Traffic congestion on highways is a multi-sectoral phenomenon affecting society, the economy and the environment. It often takes place at specific locations such as on and off-ramps, weaving segments and intersections. The on-ramp merging procedure is considered as one of the main factors that causes traffic congestion on highways. The studies in literature show that the merging procedure can result in adverse traffic scenarios such as the buildup of the vehicles on the ramp which causes a downstream drop in capacity and subsequent blockage of upstream off-ramp traffic flow. Moreover, the on-ramp vehicles need to take the actions of leading and following mainlane vehicles into account during the merging process. On highly congested roads, this merging process becomes even more tedious and undesirable stop-and-go traffic behavior becomes unavoidable. Connected and autonomous vehicles (CAVs) that can provide safe gaps between vehicles along with identifying appropriate merging speed profiles have the potential to reduce traffic accidents and improve traffic efficiency. This thesis introduces a nonlinear model predictive control (NMPC) strategy for autonomous merging control based on a cost function that tracks the desired inter-vehicular gaps for on-ramp and mainlane vehicles, and thus intends to fully exploit the capacity of the road in order to maximize the traffic throughput. The proposed controller aims to optimize both acceleration and steering rate profiles of vehicles, and to guide on-ramp vehicles to merge efficiently, without frequent slowdown or wait for merging gaps at the end of the ramp along with minimal disruption to the mainlane traffic flow. The controller is evaluated under different initial conditions, ranging from low to high traffic conditions. The performance of the controller is compared to that of a baseline scenario, and the results show that the proposed controller increases travel times in the range of 2.46% and 4.17% for different traffic conditions, without disrupting the mainline traffic operation. Additionally, average speed of vehicles is improved in the range of 8.2% and 4.5% under different traffic conditions.

Autonomous Underwater Vehicles (AUVs) are unmanned vehicles that give the opportunity to carry out lengthy and dangerous tasks autonomously. This is particularly useful for survey tasks, where the objective is to search the seafloor for objects. In this thesis work a planning system is developed that can plan a path for survey tasks, while considering environmental challenges such as communication limitations and location uncertainty. To compensate for location uncertainty, the planning system requires a higher level of abstraction compared to conventional path planning algorithms. For that reason, the planning problem is modelled in the PDDL, creating a powerful and flexible planning system which deals with the complex survey problem. Besides that, some additional planners are added to support the PDDL-planner and provide suitable plans for the AUV to carry out. The resulting plans are evaluated by simulation, showing that the planning system can successfully survey different scenarios. Besides that, the PDDL model is validated by means of the Event-B formal method, in order to obtain mathematical proofs of the validity of the planning model. The results are a step forward in achieving full autonomy of the AUVs. Besides that, a demonstration of the applicability of PDDL in real-world problems is given.

Autonomous Underwater Vehicles (AUVs) are unmanned vehicles that are often used for searching an area of the seabed for objects, such as naval mines. For autonomous planning of such search operations, it is useful to be able to infer what tasks an AUV can perform and how well it can do so. The objective for this final thesis project therefore was to develop and implement a semantic knowledge model in the form of an ontology for assessing the capabilities of AUVs that perform mine search operations. For developing an ontology, first the use case and existing ontologies for related topics were studied to obtain an overview of the concepts and relationships that are most relevant for AUVs that perform mine search operations. The ontology was then developed by formalising these concepts and relationships, and by identifying instances of these concepts that are specific to the use case. By implementing this ontology and developing a reasoning process for it, automatic inference of the capabilities of an AUV based on the components it is equipped with was realised. In addition, a data-driven methodology for performance assessment was developed and incorporated in the ontology to provide a basis for automatic inference of the expected performance of an AUV. This performance assessment methodology was tested by performing simulations with an AUV in a virtual underwater environment. These simulations showed that the developed methodology can successfully be used to predict performance under different conditions.

Renewable energy sources, e.g. solar energy and wind energy, have gained popularity as an alternative means of energy production as they do not reinforce global warming. In addition, more and more electrical appliances (e.g. electric vehicles, induction cookers, and heat pumps) are used as a substitute for appliances that need non-renewable energy sources. This increase in the use of renewable energy resources pushes the electricity grid to its limits due to new induced load peaks. The grid is not designed for these developments and as a result, asset deterioration, higher transport losses, and outages are expected to occur. The most straightforward solution for the distributed system operator, i.e. the operating manager of the distribution network, is to expand the grid. However, grid expansion is a costly operation and there are additional promising methods to decrease grid load peaks, e.g. by using different charging strategies for electric vehicles. The conventional charging strategy for electric vehicles is uncontrolled charging. With uncontrolled charging, the charging of the electric vehicle immediately commences once a connection with the charging pole is established. The smart charging strategy, however, is able to delay the charging moment to a more optimal time instant in view of, e.g. variable electricity prices. The vehicle-to-home (V2H) charging strategy is similar to smart charging, but in addition, the V2H strategy allows the electric vehicle to discharge electricity to power a nearby residential home. This research aims to compare smart charging and V2H charging on their economical effects for their users. The charging strategies are implemented using two control algorithms: a rule-based controller and a model predictive control (MPC) algorithm. The rule-based controller is implemented due to its simplicity and the MPC algorithm is used for its ability to take into account predictions of system related variables, e.g. household loads. The MPC algorithm is implemented with two different forecasts namely, perfect information, i.e. uncertain variables are forecasted perfectly, and certainty equivalent, i.e. uncertain variables are predicted using a persistence forecast model. The persistence forecast model assumes that the future values of an uncertain variable remain equal to the latest measurements, e.g. the solar generation of tomorrow is expected to be equal to that of today. The control problem is non-linear as an electric vehicle behaves differently depending on its status, e.g. driving or charging. The control problem is therefore reformulated into a mixed logical dynamical framework such that it can be solved efficiently using mixed integer linear programming. An extensive comparison in performance for a microgrid case study is done using real data of solar generation, electric vehicles, and household loads for simulation. The results show that the V2H charging strategy can outperform smart charging by reducing both the peak loads and the electricity costs. However, the V2H strategy only gives a minor extra decrease in costs compared to smart charging and the performance of V2H charging is highly dependent on the quality of the forecasts. Therefore, it is concluded that, in practice, smart charging is the most effective charging strategy. Further research is done to investigate whether the microgrid costs, using a smart charging strategy, can be reduced further by taking into account the uncertainty of some variables such as the electricity price and the household load. This is implemented through a scenario-based MPC algorithm due to its ability to incorporate multiple forecasts, i.e. scenarios, for each uncertain variable. Six different scenario generation methods are implemented which are distinguished by two characteristics: the period from which the historical error between the certainty equivalent case and the true realization of the uncertain variable is collected (i.e. yearly, seasonal, or daily) and the addition method of these errors to the persistence forecast model to generate new scenarios, i.e. as a variable or as a constant. An extensive comparison in performance for a microgrid case study is done. The results show that the certainty equivalent MPC case can be outperformed if a low number of scenarios is generated. This is achieved most effectively by collecting the persistence forecast model error from a rolling horizon of the past 24 hours and adding the error as a constant to the persistence forecast model.

Traffic signal controllers have been around since the late sixties of the 19th century and since then they have played an increasing role in (urban) traffic management. Over the years many different algorithms have been developed and in Dutch practice actuated control is mostly used. Actuated control and other similar approaches apply variable green times to handle demand fluctuation, which leads to a reduction of the delay for vehicles in comparison to e.g. fixed-time control, however, in actuated control it is difficult to accurately predict control decisions. This study proposes a new traffic signal control algorithm that creates a planning of control decisions based on expected vehicle arrivals. By doing so, predictability of control decisions increases, while no sacrifices need to be made regarding the optimality of the control policy. Since certain errors such as inaccurate queue discharge rates are incorporated in the planning of control decisions, a second algorithm is required that adjusts the planning of control decisions to actual traffic conditions. Using an N65 case study intersection the performance of the new traffic signal control algorithm is compared to other algorithms. Depending on the availability of vehicle arrivals and the composition of traffic flows at the intersection, the new algorithm outperforms standard Dutch actuated traffic signal control in terms of overall delay by 3.1% to 4.7%. A range of improvement possibilities is suggested to ensure a greater reduction of the delay.