The Learning of Linear Dynamical State Space Models According to the Principles of Free Energy

The Learning of Linear Dynamical State Space Models According to the Principles of Free Energy: The improvement of acceptance and utilization of the Machine Learning technique: Expectation Maximization in Control theory

Groeneweg, Julia (TU Delft Mechanical, Maritime and Materials Engineering; TU Delft Delft Center for Systems and Control; TU Delft Biomechanical Engineering)

Wisse, Martijn (mentor)
van den Boom, Ton (mentor)

Delft University of Technology

2018-11-14

The Free Energy principle represents a Neuroscience theory that unlike any other theory can explain all behavioral aspects of adaptive agents (like humans). The issue of behavior is made computationally tractable by considering behavioral processes as various optimization problems with a single objective: minimization of Free Energy. The rationale is that to exist adaptive agents have to resist a natural tendency to disorder. This is accomplished by adopting a policy that minimizes Surprisal. To put it differently the adaptive agent has to become an expert at predicting its own sensations (eyes and proprioception) caused by an uncertain environment. The brain is therefore believed to embody a generative model. Surprisal is the measure of how good this generative model can explain the sensations experienced by adaptive agent. The mathematical equivalent of Surprisal is the negative log Likelihood. The Likelihood distribution -a typical term used in probability theory- expresses how probable the observed sensory data -sensations- is for different settings of causes. By maximization of the Likelihood Surprisal is minimized. To obtain an exact maximum Likelihood solution one however has to integrate over all possible settings of all known and unknown causes. Rather than minimization of Surprisal a minimization of a bound on Surprisal: the Free Energy is considered. The Free Energy principle formulated into a Bio-inspired Control algorithm could be the solution to the lack of adaptability and precision towards uncertain and unknown environments that the robotic community currently faces. This formulation is however not trivial. This thesis addresses the process of Learning a model of a Process according to the principles of Free Energy. It is identified the Free Energy principle to heavenly rely upon concepts from the Machine Learning methodology Expectation Maximization. In case of linearity and a Wiener process assumption on the noise affecting the Process, the Free Energy Learning algorithm is equivalent to the EM algorithm. An evaluation of the minimization of Free Energy problem with respect to the state led to a state estimation computation equivalent to the Kalman filter. An evaluation of the minimization of the Free Energy problem with respect to the model parameters led to a model parameter computation equivalent to the solution of a linear least square problem. It has been identified the EM algorithm to be quite unfamiliar in Control settings. Implementations of the Free Energy Learning algorithm = EM algorithm yielded promising results although only a naive version has been implemented. Improvements on the implementations are believed to generate a competitive algorithm for solving control problems. Next steps are consideration of the full Free Energy principle algorithm where the Wiener assumption on noise does not apply and the input is derived with a similar Free Energy minimization problem. The former could relate to the concept of incorporating a prediction horizon suggesting a intimate relationship with the Subspace Identification method. The latter is inherently different from conventional control. The full Free Energy principle algorithm does distinguishes itself from the EM algorithm.

The Free Energy Principle
Variational Free Energy
Bayesian Inference
variational Expectation Maximization

http://resolver.tudelft.nl/uuid:e752fb8d-080b-4215-9550-ed8c3f5170f2

Student theses

master thesis

© 2018 Julia Groeneweg