Quadrupedal robots possess the ability to move freely in the world and perform a variety of actions that would be unsafe or impractical for humans to perform. In the SNOW project, a quadrupedal robot is tasked with aiding firefighters in rescue missions during house fires by locating humans and assessing their health. Pushing away obstacles that cannot be circumvented otherwise is one of the many capabilities a quadrupedal robot should possess to be of most use in such missions. We develop a proof of concept by solving two problems sequentially: which stance to take on prior to the push and how to perform the push. The process of stance selection starts with generating a certain amount of stances. Stances are generated starting from a preselected stance appropriate to the goal location and deviating from the 12 joint angles with a normal distribution. All generated stances are ran through a number of filters, which rely on solving for the forward and inverse kinematics of the robot. These filters check if the initial position is sensible and balanced and if the projected final position is close to the goal and balanced. The inverse kinematics are solved using Adaptive-Network-Based Fuzzy Inference Systems (ANFIS), which results in accurate estimations within a time frame that can be used in real-time applications. The final stance is selected by comparing the total displacement of all joint angles per stance, where the lowest total displacement is considered optimal. The push is controlled by a nonlinear model predictive controller. We strive for a stable push, where the contact between the pusher and the object sticks, by keeping the movement of the end-effector within the motion cone. The motion cone denotes all twists the object can have without slipping at the contact with the pusher and is constructed using the limit surface to model the interaction between the object and the support surface and the generalized friction cone to model the interaction between the pusher and the object. We find that the motion cone predicts stick and slip with an accuracy slightly higher than 80%. Our controller steers accurately to all goals that lie within the motion cone and moves the object with a twist on the edge of the motion cone if the goal location lies outside of the motion cone. The robot remains balanced throughout the pushing motion in the vast majority of cases, but is more at risk of tipping over when pushing heavier objects.

To operate in open world environments a symbolic Artificial Intelligence (AI) to be able to adapt and incorporate new objects and relations in its Knowledge Base (KB). Symbolic AI use the objects and relations in their KB to navigate the world and create plans. These KB are filled with knowledge in advance, so they can not add new knowledge when encountering novel situations. This thesis presents Heuristics-Based Causal Discovery (HBCD), a method that identifies and labels causal relations and transforms those causal relations into logic statements, which can be inserted into a KB autonomously. HBCD can operate in a partially-observable environment by performing actions and observing the effects of its actions. The actions are chosen by heuristics, which are modelled after human strategies for causal discovery. The discovered causal relations are tested for the properties necessity and sufficiency. These properties provide information on the completeness of the result, whether there are any missing causal relations. HBCD uses this information to adjust its search space by moving variables in and out of it during the search. If there are no more missing causal relations HBCD stops the search. The method was tested in two simulated environments and the results are promising.

During the whole history of Computer Science as we know it, the end goal has been to solve problems faster and more efficiently than us humans. Computers came to be to perform repetitive tasks we do not want to or do not have time to perform ourselves. At some of them, such as simple mathematical calculations, they were and they are indeed better and faster than a human being. With the appearance of computers, some effort was also put in the study of the human brain and the idea of building machines with similar principles of operation: The idea of emulating conscience has always been attractive to humankind.

In this work, we address the challenges of employing robots in the Search-and-Rescue (SAR) domain, where they can benefit rescue workers to quickly obtain Situational Awareness (SA). Missions with autonomous mobile robots are heavily dependent on environmental representations. Representations have been steadily increasing in the richness that they can capture, in addition to geometry we can now represent objects and their interrelations. These richer environmental representations, such as the 3D scene graph, have provided an opportunity to integrate representations with prior knowledge to make them more actionable and improve the SA they provide. The use of such representations for planning is limited. In previous work, the main approach was to augment and extend the scene graph to enable traditional symbolic planning. The limitations of these methods are that they are offline, scale poorly, and require full observability, making them unsuitable for the SAR domain. The main contributions of this work are as follows. First, we propose the behavior-oriented situational graph, a data structure that integrates data-driven perception with prior knowledge following a novel situational affordance schema. This schema connects situations with robot behaviors and mission objectives, allowing for autonomous mission planning. Second, we propose an efficient method to obtain task utilities from the proposed behavior-oriented situational graph through planning. Finally, we propose an exploration component to discover and select tasks online in dynamic environments that are potentially partially observable. This work is evaluated in several simulation scenarios, showing improved efficiency in mission completion compared to offline methods for the specific SAR domain. Finally, the methods are implemented and tested in a real-world scenario using a mobile Spot robot, showing its effectiveness in practice.

In this thesis, an autonomous multi-robot system for indoor exploration in limited network environments is proposed. The specific use case is search and rescue where the operators must have access to the most up-to-date information, necessitating the requirement for communication maintenance. This requirement is satisfied in a novel way by using signal strength measurements collected by the robots to update the network model, thus reducing the overly conservative nature of current approaches. The network model chosen is an adaptation of existing path loss models as they balance complexity and performance given the information available during multi-robot exploration. The proposed system consists of a central server with multiple robots which perform tasks that are readily distributable, whereas the central server performs complex iterative optimizations such as merging the local maps and running task assignment. While fully distributed architectures offer theoretical benefits such as scalability and robustness, in practice, these benefits are currently not achieved due to the complexity of splitting iterative optimizations such as mapping and task assignment among multiple nodes. This results in distributed architectures requiring more time and bandwidth to solve these problems, which is why this work implements a central architecture. While iterative optimizations are more efficient when performed centrally, scalability is limited due to the branching of possible options during task assignment. To combat this, an action space formulation, called an action graph, is proposed that is unique to each robot and reduces the number of actions by merging similar ones. Both the dynamic network prediction model and the action graph formulation are shown to hold promise by experimentation. However, more work is needed before real-world use is feasible.

In this thesis, an autonomous multi-robot system for indoor exploration in limited network environments is proposed. The specific use case is search and rescue where the operators must have access to the most up-to-date information, necessitating the requirement for communication maintenance. This requirement is satisfied in a novel way by using signal strength measurements collected by the robots to update the network model, thus reducing the overly conservative nature of current approaches. The network model chosen is an adaptation of existing path loss models as they balance complexity and performance given the information available during multi-robot exploration. The proposed system consists of a central server with multiple robots which perform tasks that are readily distributable, whereas the central server performs complex iterative optimizations such as merging the local maps and running task assignment. While fully distributed architectures offer theoretical benefits such as scalability and robustness, in practice, these benefits are currently not achieved due to the complexity of splitting iterative optimizations such as mapping and task assignment among multiple nodes. This results in distributed architectures requiring more time and bandwidth to solve these problems, which is why this work implements a central architecture. While iterative optimizations are more efficient when performed centrally, scalability is limited due to the branching of possible options during task assignment. To combat this, an action space formulation, called an action graph, is proposed that is unique to each robot and reduces the number of actions by merging similar ones. Both the dynamic network prediction model and the action graph formulation are shown to hold promise by experimentation. However, more work is needed before real-world use is feasible.

In research, the overall power production in a wind farm is typically increased by employing model-based wind farm control. A controller, in an open-loop setting, operates based on the velocity of the wind flow, predicted by a wind farm model. For a controller to achieve the desired level of power production, a wind farm model has to be accurate and computationally tractable. Generally, high-fidelity wind farm models are accurate but are computationally complex, making real-time control infeasible. This issue can, however, be addressed by employing closed-loop approach. In this approach, low- or medium-fidelity wind farm models, which are computationally tractable, are used, and their accuracy Is improved by employing an estimator. In the previous researches, a centralized estimation approach was employed. In this framework, a single estimator is employed for estimating all the states (second-to-second wind field at turbine hub height) in a wind farm. Simulation results show that the accuracy of an open-loop model can be improved. However, the problem is the state size of the wind farm models. This leads to the objective of the thesis, which is “Can the accuracy of a wind farm model be improved while maintaining computational tractability?”. In this thesis, distributed estimation is proposed as a solution to this problem. The basic idea behind distributed estimation is to distribute a wind farm into a number of small spatial domains (subsystems), and define a wind farm model for each of these subsystems to independently predict the wind flow in their respective spatial domain. For estimation, each sub-system employs an estimator to independently estimate their respective states, in parallel. In this thesis, based on the extent to which the states are estimated (measurement-update), model distribution, and size of the subsystems, four types of distributed architectures are devised, using the medium-fidelity model WindFarmSimulator (WFSim). Simulations show negligible loss in performance, and at the same time, the time taken for each iteration decreases drastically, making it computationally tractable. In conclusion, distributed architectures are capable of improving the accuracy of the open-loop wind farm models, to the same level of accuracy offered by the centralized architecture, while maintaining computational tractability. Additionally, application of these distributed architectures for controller design will be a scope for future research in this topic.

This master thesis introduces Hierarchical Active Inference Control (HAIC) as a control method for nonholonomic systems. This method only requires tuning of a minimal number of hyperparameters and has a relative low computation load. HAIC is based on recent research done in the application of the neuroscientific theory of Active Inference for robot control. The hierarchical aspect of this method introduces two layers of Active Inference Control (AIC) and the introduction of an additional decision hyperparameter. The top layer AIC is designed such that it takes the nonholonomic constraint into account. This causes HAIC to be able to control a nonholonomic mobile robot to a three-dimensional reference state in comparison to regular AIC which is unable to. The selection of the introduced hyperparameter gives a trade-off between robustness against noise present in the measurements and possible error in the y-dimension of the local reference frame. Convergence of a two-wheeled differential-drive mobile robot to a reference state using HAIC is demonstrated both by simulation and physical experiment. Simulations are done for a large range of different starting states. These simulations show the ability of HAIC to converge the mobile robot for any starting state. It also confirms that a higher value for the decision parameter causes a larger error. Another group of simulations is done investigating the new hyperparameter introduced with HAIC. These simulations check the ability to converge the system when different noise levels are present in the measurements. It is concluded that tuning of the newly introduced hyperparameter needs to accommodate the noise present in the measurements. Where the higher the noise, the higher the value of the hyperparameter is needed for convergence. A physical experiment is done where the robot is controlled to five different reference states. Data obtained from this experiment shows the ability of HAIC to control a physical nonholonomic system. Like the simulations, it is also shown that a correct value is required to set the newly introduced hyperparameter. Additionally, the physical experiment shows that HAIC is not robust when disturbances can cause sudden large value changes in the measurements.

Reinforcement learning (RL) is an area of Machine Learning (ML) concerned with learning how a software-defined agent should act in an environment to maximize the rewards. Similar to many ML methods, RL suffers from the curse of dimensionality, the exponential increase in solution space with the increase in problem dimensions. Learning the hierarchy present in underlying problems, formulated using the Markov Decision Processes (MDPs) framework, may exploit inherent structure in the environment. Using the hierarchical structure, an MDP can be divided into several simpler semi-MDPs (SMDPs) having temporally extended actions. The solutions of smaller SMDPs can then be re-combined to form a solution for the original MDP. The methods for Hierarchical Reinforcement Learning (HRL) explore ways to break down the original problem into SMDPs while providing several opportunities for state and temporal abstractions. A novel algorithm for learning this hierarchical structure of a discrete-state goal-oriented Factored-MDP (FMDP) is proposed in the thesis work taking into account the causal structure of the problem domain with the use of Dynamic Bayesian Network (DBN) model. The proposed method autonomously learns the state and temporal abstractions in the problem domain and constructs a hierarchy of SMDPs using them. Such a decomposition results in decreasing the problem state dimensions to be considered for solving each SMDP and, hence, reducing the computational complexity induced due to increased dimensionality.

Autonomous underwater vehicles (AUVs) are unmanned vehicles that operate underwater. These vehicles can be used for various operations, including scanning the ocean floor in order to search for objects. In such operations, the AUV navigates close to the ocean floor, using its sonar systems to map this ocean floor. Before an AUV is launched on a search operation, usually very little is known about the underwater conditions at the search location. Currently, the AUV’s coverage path is often defined before the start of the operation. However, the coverage path depends on the quality of the sonar images, which in turn depend on the unknown and varying underwater conditions. Therefore, it is unlikely that the predefined coverage path is optimal. In this final thesis project a coverage path planner is developed, which autonomously adapts the AUV’s coverage path to the changing underwater conditions during an operation. Since the radio waves carrying the global positioning system (GPS) signal do not travel far in seawater, the AUV needs to keep track of its location using an inertial navigation system (INS). Unfortunately, the error on the location measurements obtained from an INS grows over time, causing the AUV’s believed trajectory to differ significantly from the AUV’s true trajectory. As a result, parts of the search area might be believed to have been covered, while these parts have in reality not been visited. Taking the AUV’s uncertain trajectory into account thus increases the complexity of the coverage path planning problem. In order to keep track of which parts of the search area have been covered, and to which extent, a coverage map of the search area is maintained during an operation. This coverage map is constructed such that it takes the uncertainty on the AUV’s location measurements into account. Three different coverage path planners have been developed that use this coverage map to autonomously plan the AUV’s coverage path during an operation. The three different coverage path planners have been implemented in MATLAB, and they have been evaluated in a simulation framework developed by TNO. The simulation results show that all three coverage path planners consistently achieve the required coverage of the search area, in a scenario where no large changes in the underwater conditions occurred. Furthermore, one of the coverage path planners has been evaluated in three scenarios where large changes during the operation did occur. Halfway during these scenarios, either the strength of the underwater currents increased, the required coverage of the search area increased, or one of the AUV’s sonar systems broke down. In all of these scenarios, the coverage path planner successfully adapted the coverage path such that the required coverage of the search area was still achieved.

After the Second World War, chemical warfare agents and munitions were dumped in the Baltic Sea and the North Sea. In order to assess the severity of the environmental consequences, it is important that the chemical warfare agents are located and their condition is investigated as soon as possible. In order to reduce this time, the search could be performed by Autonomous Underwater Vehicles (AUVs). The goal of this thesis is to develop algorithms that can be applied during underwater operations to allow AUVs to optimize their actions based on a global objective function without centralized communications. The search problem is modeled within the Distributed Constraint Optimization Problem (DCOP) framework to be able to explicitly define both computational agents and their communications. In order to be applicable to AUV operations, both benchmark problems and real-world problems with continuous domains are modelled within the Continuous DCOP (C-DCOP) framework. This preserves the flexibility of modeling inherent in a DCOP while removing the limitations imposed by the discrete definitions. Two C-DCOP algorithms are presented in this thesis. The Compression-DPOP (C-DPOP) algorithm discretizes the domain of each of the variables and compresses their domains in order to refine the search space at every iteration. The Distributed Bayesian (D-Bay) algorithm leverages Bayesian optimization to solve C-DCOPs without any need for discretization by modelling the effects of the variables on the global utility as Gaussian processes. Results from high-fidelity simulations and real-world experiments are given for real-world multi-agent search problems. A mine countermeasures operation is simulated in which AUVs update their search areas during the search based on sonar performance. Assigned areas are re-distributed in order to optimize metrics relating to the expected time of completion and the level of confidence that all mine-like objects have been detected. Moreover, real-world experimental results are presented for a multi Unmanned Aerial Vehicle (UAV) search problem. By improving the autonomy of AUVs, the search efficiency can be increased through the cooperative optimization of their actions during the operation. The research in this thesis contributes to this strategy by means of the developed algorithms and their applicability to real-world problems.@en

Energy systems influence many aspects of society, from the residential sector to the commercial one. Improving the performance and efficiency of energy systems and guaranteeing their stability is a fundamental task of control engineers. In this regard, this thesis presents modeling and control solutions for energy systems, with a focus on both electric and thermal ones. The thesis is divided in three parts. Firstly, we consider an online partitioning and stability problem of a network applied to frequency regulation. Secondly, we present algorithms for energy management system of an electrical microgrid. In particular, we focus on providing a trade-off between computational complexity and performance of the obtained solution. Lastly, we focus on thermal energy systems by designing an algorithm for room temperature control in commercial buildings. In the first part of the thesis, we consider a linear switching large-scale system and we focus on the problem of partitioning the system into smaller subsystems. We assume that the different modes of the switching system are not known a priori, but they can be detected. We propose an online scheme that can partition the system when the mode switches, adapting therefore the partition to the mode of the switching system. The goal of the partitioning algorithmis on the one hand to minimize the coupling between subsystems, in order to facilitate the task of a distributed/decentralized controller, and on the other hand to obtain subsystems with similar sizes, in order to distribute the control effort equally. Moreover, after the system has been partitioned, we apply a decentralized state-feedback control scheme to stabilize the overall system. In order to prove stability, we apply a dwell time stability scheme such that the closed-loop system remains stable even after both the mode and partition changes. The online partitioning method, together with the control algorithm, is applied to an automatic generation control problem of frequency regulation in a large-scale power network. In the second part of the thesis,we consider the energy management system problem in a microgrid. We present several Model Predictive Control (MPC) approaches for optimally managing the power flows in the microgrid, from an economical point of view. The microgrid is modeled using the Mixed Logical Dynamical (MLD) framework. We provide three different strategies that yield a trade-off between computational complexity and performance by parameterizing the inputs to the system. First, we propose a parametric MPC approach, in which the continuous inputs are expressed as parametric functions and the binary variables are heuristically parameterized. Next, we propose an if-then-else parametrization of the binary variables in the MLD model, so that they are assigned a value before the optimization takes place, yielding therefore a real-valued optimization instead of a mixed-integer one. Finally, we use past optimization results obtained from simulations to develop two machine learning methods, i.e. decision trees and random forests, that can provide a binary variable configuration so as to, once again, remove the binary variables from the optimization problem. The results obtained show that the methods can provide a very large decrease in computation time while having almost no loss in performance. Simulation results show how the developed methods are able to provide a large reduction in computation time while having a very little performance loss. Lastly, in the third part we focus on thermal networks. We propose a scenario-based MPC approach to control the temperature room in office buildings. The building is modeled using the tool Modelica that yields a better model description compared to linearized models. The adopted scenario generation method improves upon the current literature by considering that the marginal distributions depend both the prediction time steps and on time itself and that the distributions of the disturbances are not stationary. By combining scenario-based MPC together with Modelica, we can improve the performance of the controller of the building and we show this by comparing our method against a deterministic method using a Modelica model description, but also against the same controllers with a linearized model. @en