Active inference is a neuroscientific theory, which states that all living systems (e.g. the human brain) minimize a quantity termed the free energy. By minimizing this free energy, living systems keep an accurate representation of the world in their internal model (learning), are provided with an optimal way of acting on the world (action selection), and can predict incoming sensory data (perception). Considering the fact that these properties are sought after in artificial intelligence systems as well, active inference has also become an interesting topic from an engineering point of view. The application of active inference can be done with both a continuous and a discrete state-space framework. However, research on discrete state-space active inference has neglected the extension of its applicability to nonstationary environments. This work aims to fill that gap. More specifically, the goal of this research is to evaluate the performance of state-of-the-art discrete state-space active inference agents in nonstationary environments, and assess whether forgetting part of the agent's previous experiences can increase its performance. The type of nonstationarity that is used in this work is cyclostationarity, and this nonstationarity will only be manifested in the transition process of the active inference task. Moreover, the specific type of task solved is one of planning and navigation in a gridworld. Since the agent has to deal with a planning and navigation task, performance is quantified by the number of steps the agent needs to take in order to reach its goal. Three methods of forgetting are implemented and compared, inspired by techniques from reinforcement learning, deep learning and time series analysis respectively. These are: (1) the use of a constant forget rate, (2) the use of the updating mechanism of a long short-term memory (LSTM) cell applied to the updating of the generative model in active inference, and (3) the use of a memory window that stores experience only from a certain trial onwards and forgets experience from before this trial by utilizing a rolling summation of the concentration parameters. The results show that forgetting with the use of a memory window can significantly improve performance, provided that the agent can reach the goal state from the initial state in one trial. When this is not the case, the use of a memory window does not (positively or negatively) influence performance. Both the implementation of forgetting based on the updating of an LSTM cell and the use of a constant forget rate have unanimously shown to decrease performance, and thus should not be implemented in active inference.

The free energy principle is a recent theory that originates from the neuroscience. It provides a unified framework that combines action perception and learning in the human brain. This research aims to implement the perception aspect of the free energy principle into robotics. This is achieved via the dynamic expectation maximisation (DEM) algorithm. DEM is derived from the free energy principle and provides a novel solution for state and input observation for LTI systems under the influence of coloured noise. This thesis provides an overview of how the DEM observer is derived from the free energy principle. Thereafter, an experimental design is presented in which data is collected to validate the performance of DEM on experimental data. The data is collected from a quadcopter drone flying in wind conditions. A detailed overview of the DEM observer settings is presented and motivated. \\ This research shows that because of its use of generalized coordinates DEM is able to leverage the coloured nature of the noise for better state estimation. This is demonstrated by the fact that DEM obtains a higher state estimation accuracy than other coloured noise state observers, such as state augmentation and SMIKF. Moreover, in input estimation, DEM is able to obtain similar results as an unknown input observer. Finally, the accuracy vs complexity trade off of DEM is highlighted.

Recent developments in neuroscience research, mainly introduced by neuroscientist Karl J. Friston, have resulted in a concept called the Free Energy Principle (FEP). The FEP is a brain theory unifying action, perception and learning. An important observation is that autonomous robots have to perform similar tasks to the human brain. Therefore, these developments are very interesting from a control engineering perspective. A lot of research is going on in the direction of Active Inference (AI), which includes the perception and action parts of the FEP used for filtering and control on physical systems. However, a research gap exists for the theory describing the complex mathematical backbone of perception and learning, called Dynamic Expectation Maximization (DEM). DEM is a parameter estimation algorithm that can be used to perform filtering as well as system identification. This thesis is part of the research aiming to evaluate the performance of DEM as a filtering as well as system identification technique. To be more specific, this thesis considers the filtering part. In general, a filter is meant to derive the states of a system using a system model and sensor measurements. The system model is often not perfect, resulting in process noise, and the sensors provide noisy data, resulting in measurement noise. Many existing filtering techniques, including the conventional Kalman filter, assume these noises to be white. However, the process noise contains unmodelled system dynamics that introduce correlation. Therefore, by definition, the process noise is not white, but coloured. The main advantage of using the DEM filter is the ability to extract information from the correlation in the measurements and the noises to construct a better state estimate. Simulation results in previous work indicated the potential of DEM as a filter. The next step is to prove the usefulness of the DEM filter on a physical system. In order to do so, this thesis describes the design of an experimental setup with a quadrotor Unmanned Aerial Vehicle (UAV) used to evaluate the performance of the DEM filter. The recorded flight data is used to analyze the process and measurement noises. Furthermore, the DEM filter is compared with the conventional Kalman filter. It turns out that for a relatively big subspace of DEM tuning parameters, the DEM filter is able to outperform the Kalman filter, which proves the potential of this filtering algorithm in a practical robotics setting.