The Free Energy Principle, which underlies Active Inference (AI), is a way to explain human perception and behaviour. Previous literature has hinted at a relation between AI and Linear-Quadratic Gaussian (LQG) control, the latter being a textbook controller. AI and LQG are, however, defined with different settings in mind: LQG has access to inputs, whereas AI estimates these; LQG is optimal for White Gaussian Noise, whereas noise needs to be coloured for AI, in order to make derivatives. Therefore, a comparison is provided on two bases: the setting of LQG control; and the setting of AI. The optimal LQG controller is obtained for both settings, and AI is applied to both settings as well. When AI is reduced to the setting of LQG, an equivalent expression can be obtained by a proper choice of tuning parameters. This entails choosing a matrix such that the closed-loop is stable, which contrasts LQG control, which is always a stabilizing controller. When LQG is extended closer to the normal AI setting, we find that tuning of AI becomes harder for more complex systems, but that AI is mostly able to show optimal behaviour.

Active inference is a neuroscientific theory, which states that all living systems (e.g. the human brain) minimize a quantity termed the free energy. By minimizing this free energy, living systems keep an accurate representation of the world in their internal model (learning), are provided with an optimal way of acting on the world (action selection), and can predict incoming sensory data (perception). Considering the fact that these properties are sought after in artificial intelligence systems as well, active inference has also become an interesting topic from an engineering point of view. The application of active inference can be done with both a continuous and a discrete state-space framework. However, research on discrete state-space active inference has neglected the extension of its applicability to nonstationary environments. This work aims to fill that gap. More specifically, the goal of this research is to evaluate the performance of state-of-the-art discrete state-space active inference agents in nonstationary environments, and assess whether forgetting part of the agent's previous experiences can increase its performance. The type of nonstationarity that is used in this work is cyclostationarity, and this nonstationarity will only be manifested in the transition process of the active inference task. Moreover, the specific type of task solved is one of planning and navigation in a gridworld. Since the agent has to deal with a planning and navigation task, performance is quantified by the number of steps the agent needs to take in order to reach its goal. Three methods of forgetting are implemented and compared, inspired by techniques from reinforcement learning, deep learning and time series analysis respectively. These are: (1) the use of a constant forget rate, (2) the use of the updating mechanism of a long short-term memory (LSTM) cell applied to the updating of the generative model in active inference, and (3) the use of a memory window that stores experience only from a certain trial onwards and forgets experience from before this trial by utilizing a rolling summation of the concentration parameters. The results show that forgetting with the use of a memory window can significantly improve performance, provided that the agent can reach the goal state from the initial state in one trial. When this is not the case, the use of a memory window does not (positively or negatively) influence performance. Both the implementation of forgetting based on the updating of an LSTM cell and the use of a constant forget rate have unanimously shown to decrease performance, and thus should not be implemented in active inference.

The free energy principle is a recent theory that originates from the neuroscience. It provides a unified framework that combines action perception and learning in the human brain. This research aims to implement the perception aspect of the free energy principle into robotics. This is achieved via the dynamic expectation maximisation (DEM) algorithm. DEM is derived from the free energy principle and provides a novel solution for state and input observation for LTI systems under the influence of coloured noise. This thesis provides an overview of how the DEM observer is derived from the free energy principle. Thereafter, an experimental design is presented in which data is collected to validate the performance of DEM on experimental data. The data is collected from a quadcopter drone flying in wind conditions. A detailed overview of the DEM observer settings is presented and motivated. \\ This research shows that because of its use of generalized coordinates DEM is able to leverage the coloured nature of the noise for better state estimation. This is demonstrated by the fact that DEM obtains a higher state estimation accuracy than other coloured noise state observers, such as state augmentation and SMIKF. Moreover, in input estimation, DEM is able to obtain similar results as an unknown input observer. Finally, the accuracy vs complexity trade off of DEM is highlighted.

A fundamental task of intelligent and autonomous robots is to infer from observations the state of the world. This inference is generally achieved by employing a filter, which consists of a model and filtering law. Learning this model and filtering law from observations is another fundamental part of robotics, and is generally referred to as system identification.

Neuroscientist K.J. Friston has developed a relatively novel theory on biologically plausible human brain inference called the Free-Energy Principle. One of the theories within the Free-Energy Principle, namely that Dynamic Expectation Maximization (DEM), has been suggested as a novel method for filtering and system identification. This method is expected to outperform standard Expectation Maximization (EM) in terms of hidden state and parameter estimation in settings where noise is correlated. However, in order for this neuroscientific theory to be properly used for robot inference, two problems must first be solved.

The first of these problems is the fact that the theory is defined in the continuous-time domain, whereas data available for system identification is always discrete. In this thesis I will suggest three discrete-time interpretations for DEM-based system identification. The major difference between the three methods is the information that is embedded in the generalized signals: predictions, derivatives and past data.

The second problem is that the filtering method corresponding to the Free-Energy Principle depends on data which is not available: the derivative signals of measured in- and outputs. I introduce two fundamentally different solutions to this feasibility issue: a numerical differentiator and a stable filter. Both of these solutions are shown to find an estimate for the unavailable data. However, the former is shown to significantly outperform the latter.

Furthermore, the theory described in this thesis is implemented into a novel python toolbox for system identification. This toolbox can be used as a basis for further research and be approved along with it, until at some point it is ready to be used for real applications.

Using the toolbox, the DEM-based identification and filtering methods are tested though various numerical simulations and the results are compared with the EM method. Results show that with the implemented settings none of the suggested discrete-time filtering methods outperforms the conventional Kalman filter. The main cause of this inferior performance is shown to be instability in the filtering method. I make some suggestions for overcoming this problem. As a result of the inferior performance, the joint-performance of the suggested DEM-based parameter- and state- estimation methods also proves to be inferior in terms of parameter estimation accuracy.

However, results show that the theoretical parameter optima of the Free-Energy as determined from known hidden states are in fact close on the real parameters, and furthermore show to be invariant to noise correlation. This suggests that should the instability issue as some point be solved and a better means to approximate the theoretical optimum be found, the DEM-based methods might in fact outperform EM both in terms of hidden-state and parameter accuracy in settings with correlated noise.

Recent developments in neuroscience research, mainly introduced by neuroscientist Karl J. Friston, have resulted in a concept called the Free Energy Principle (FEP). The FEP is a brain theory unifying action, perception and learning. An important observation is that autonomous robots have to perform similar tasks to the human brain. Therefore, these developments are very interesting from a control engineering perspective. A lot of research is going on in the direction of Active Inference (AI), which includes the perception and action parts of the FEP used for filtering and control on physical systems. However, a research gap exists for the theory describing the complex mathematical backbone of perception and learning, called Dynamic Expectation Maximization (DEM). DEM is a parameter estimation algorithm that can be used to perform filtering as well as system identification. This thesis is part of the research aiming to evaluate the performance of DEM as a filtering as well as system identification technique. To be more specific, this thesis considers the filtering part. In general, a filter is meant to derive the states of a system using a system model and sensor measurements. The system model is often not perfect, resulting in process noise, and the sensors provide noisy data, resulting in measurement noise. Many existing filtering techniques, including the conventional Kalman filter, assume these noises to be white. However, the process noise contains unmodelled system dynamics that introduce correlation. Therefore, by definition, the process noise is not white, but coloured. The main advantage of using the DEM filter is the ability to extract information from the correlation in the measurements and the noises to construct a better state estimate. Simulation results in previous work indicated the potential of DEM as a filter. The next step is to prove the usefulness of the DEM filter on a physical system. In order to do so, this thesis describes the design of an experimental setup with a quadrotor Unmanned Aerial Vehicle (UAV) used to evaluate the performance of the DEM filter. The recorded flight data is used to analyze the process and measurement noises. Furthermore, the DEM filter is compared with the conventional Kalman filter. It turns out that for a relatively big subspace of DEM tuning parameters, the DEM filter is able to outperform the Kalman filter, which proves the potential of this filtering algorithm in a practical robotics setting.

Active inference is a process theory arising from neuroscience which casts perception, action, planning and learning under one optimisation criterion: minimisation of free energy. Current literature on the implementation of discrete state-space active inference focuses on scalability, the comparison to reinforcement learning and its performance to learn the state-observation mapping of the environment. In these implementations, active inference is supplied with accurate predefined information (knowledge) about the transition dynamics (controlled state transitions) of the environment. However, it is likely that in practical settings, knowledge gaps about controlled state transitions are involved. This thesis analyses the influence of knowledge gaps about controlled state transition on the active inference process. First, we provide a comprehensive overview of active inference and related principles. Secondly, we expand current active inference formulations with the power to actively evaluate the consequences of actions in their ability to resolve transition knowledge gaps. Thirdly, we analyse the behaviour of this expanded version of active inference once concerned with uncertain transition knowledge in a pure explorative setting (no task set). We found that active inference can deal with minor transition uncertainty, however, fails to operate when no initial transition knowledge is supplied due to a failing parameter updating mechanism. After proposing an alternative parameter updating mechanism, we found that active inference can deal with total uncertainty. Finally, we analysed the performance of active inference (with alternative parameter updating mechanism) to reach goals, starting without knowledge about transitions. We found that the resulting behaviour is a trade-off between exploration and exploitation and that the active inference agent gains transition knowledge while reaching the goals. In short, by investigating the influence of knowledge gaps in controlled state transitions, we aim to bring discrete state-space active inference one step closer to practical applications.

This research focuses on proving the presence of coloured noise on the longitudinal, lateral and rotational velocity in steady-state cornering of a skid steer mobile robot. Furthermore, it also focuses on the creation of a Gaussian filter which is able to recreate the characteristics of the measured coloured noise. This is done to determine the characteristics of the Gaussian filter based on real-life experimental values, which defines the Gaussian filter that creates coloured process noise in the Active Inference algorithm. To accomplish the main research goal, the following is done: - Create a linear dynamical model of the jackal robot and optimize the model in such a way that its velocity states resemble the experimental values of the velocities for the given experimental inputs. - Show the presence of coloured noise in the dynamical behaviour of the jackal robot. - Create a Gaussian filter and determine its characteristics, which can recreate the coloured noise found in the velocity states of the jackal robot in steady state turning. To accomplish this, experiments are done capturing the linear acceleration, rotational ve- locities, position and heading of the skid steer mobile robot using both the internal sensors of the robot as well as an external motion capturing system. A linear model of the jackal robot is constructed and discretised as the noise is a function of the difference between the experimental value for the next step state [k + 1]exp and the prediction of the next step state [k + 1]est. The prediction is made by inputting the experimental values for step [k] into the discretised linear model together with the model input. The difference between [k + 1]exp and [k+1]est is the noise on the velocities during the steady-state cornering experiments. Optimal values for Gaussian filter are found by fitting the autocorrelations of coloured noise which is made by filtering white noise with a range of Gaussian filters on the autocorrelation of the measured coloured noise. The result of this research is proof of the presence of coloured noise in skid steer mobile robots which in turn indicates that using Active Inference on these types of robots is a worthwhile approach to control and state estimation.

The trajectory tracking efficiency of a quadrotor Micro Aerial Vehicle (MAV) position controller is decreased by discrete jumps in the pose estimate provided by a localization algorithm. This paper presents a solution to this problem by first introducing a new quadrotor MAV position control architecture followed by two methods that can compensate for the jumps in the pose estimate. The new control architecture consists of Model Predictive Control (MPC) for the outer position control loop and Incremental Nonlinear Dynamic Inversion (INDI) to control the inner angular accelerations. The attitude itself is controlled with a Proportional-Derivative (PD) controller. The first method that compensates for the jumps in the pose estimate considers an adaptive control law that changes the weights in the MPC cost function when the quadrotor's position is known to be uncertain. The second method applies a filter to the reference signal before providing it to the controller. Simulations validate the proposed controller architecture and show that the adaptive controller and the controller with reference filter decrease the covered distance during a flight mission by 6.7% and 7.0%, respectively, compared to the controller without these enhancements. Therefore, the efficiency of the trajectory tracking is increased.