Matrix Capsule Networks with EM Routing, introduced by Hinton et al. (2018a), offer a powerful way to model part-whole relationships and pose information in neural networks. However, their broader adoption has been limited, possibly due in part to the complexity of the model and inconsistencies between the original paper and its official implementation. In this thesis, we present a faithful and modular reimplementation of the Matrix Capsule Network in TensorFlow 2, addressing missing and unclear aspects of the original description. Our implementation achieves competitive results on the SmallNORB dataset and is designed to be accessible, adaptable, and well-documented. By clarifying key architectural details and providing open-source code, we aim to support further research and lower the barrier to working with capsule-based models.

This thesis is inspired by Active Inference to contribute to its improvement in the Robotics work field. However, the results and applications of this thesis are useful in a broader perspective, namely in any field that makes use of derivatives and the forecasting of a time-series signal. The goal of this study is to determine a new approach for calculating the derivatives using the finite difference technique and forecasting. The finite-difference technique is based on the Taylor expansion. The Taylor expansion uses derivatives to calculate a future signal sample, while the finite difference technique uses signal samples to estimate derivatives. The accuracy of the estimation can theoretically be increased using more signal samples and choosing signal samples that are closer to the signal sample of interest. For these reasons, the central method is interesting as it maximizes both of these properties. The central method uses future signal samples for the sample of interest which are not available in an online setting. Future values can be forecast through for example statistical models and predictive coding. Statistical models such as the auto regression method are the golden standard but cannot learn in an online fashion. Predictive coding can learn in an online fashion and research found a promising branch utilizing neural network frameworks. PredNet came out as an interesting model due to its state-of-the-art results and flexibility. The research question in this thesis is as follows: can predicting future data points by use of PredNet help improve the accuracy of the generalized coordinates?
Two models of PredNet are trained on data obtained from random commands from a simulated Jackal robot. One is trained in an online fashion and one in an offline fashion. Its predictions are compared against two other forecasting methods: copying the input and the auto regression method. The predictions are then used with the finite difference method to estimate derivatives. Four methods are used: a method using as little past data as possible, a method using as much past data as possible, a method using as little centralized data as possible and a method using as much centralized data as possible. Derivatives are determined on the same simulated Jackal data and are compared against a derivative obtained from a central method using as much data as possible that all come from the true signal.

Results show no forecasting difference between the two types of PredNet models, but there are differences in training speed: training in an online fashion is not fast enough to keep up with the data stream. This suggests that the quality of neural networks is good enough for online applications, but that more work needs to be done on optimizing neural networks for online learning to make it a practical solution. However, the PredNet predictions are not as accurate as the auto regression method. This means that neural networks can still learn from the auto regression method when used for forecasting time series. An example of improving neural networks is making hybrid models.
Using the PredNet forecasts for estimating derivatives did not increase the accuracy of the derivatives meaning the predictions are not accurate enough. Besides that, using more samples, either from the past or the future, did not increase the derivative estimate either. Most of this outcome can be attributed to the choice of data set which contained many transitions between commands which could not be predicted by the PredNet models or be seen by the finite difference method.
Suggestions for improvements are to focus the training of a model on either the transition period or the period in between, using more modern models and/or using different methods for estimating derivatives.

This work concludes that neural networks show promise for online applications, that training a model requires a specific data set or a complex enough model to utilize a more general data set, and that using more samples with the finite difference method is not guaranteed to provide more accurate estimates of derivatives.

Active inference is a novel brain theory based on the free energy principle, stating that every organism, in order to stay alive, minimizes a certain free energy. This theory is being translated into robot control, hoping to mimic the capabilities of the brain. Research in this field of robotics is still quite young, and active inference has yet to mature into a reliable control theory. Implementations of active inference are scarce, and hierarchical implementations, containing multiple layers of active inference, are almost non-existent. This work presents the first implementation of hierarchical active inference performing online control on a robot in continuous state-space. The robot in question is a skid-steering mobile robot, on which two different active inference controllers are implemented and compared. The first controller is a non-hierarchical active inference controller giving wheel speed commands to the default PID controller of the robot manufacturer based on desired body velocities. The second controller is a novel implementation of a hierarchical active inference controller, that besides the function of the first controller, also replaces the low-level PID control with active inference, resulting in two-layers of active inference ultimately taking desired body velocities as inputs and translating it into open loop voltage commands for the motors. Both controllers are tuned to give roughly the same step responses in order to get a clearer view on the differences between non-hierarchical and hierarchical active inference. During the tuning process of the hierarchical controller, it became clear that no amount of tuning could get the (causal) states to convergence sufficiently fast. This problem was identified to be caused by the message passing in between the layers of active inference, where the prediction error from below negatively impacts convergence. This impact scales together with the tuning parameters, and therefore is not easily fixed by tuning alone. A potential solution is given in the form a an extra parameter in front of the upwards prediction error, giving the ability the independently tune the impact of this error on the behaviour of the system. This solution is used to fully tune the hierarchical controller, after which the performance is compared to the non-hierarchical controller, using a series of experiments. A total of 8 experiments are done where both controllers are tested performing two driving actions in two situations, namely cornering and pivoting on the ground and with the wheels suspended. The non-hierachical controller, as a baseline, performed satisfactory, but the hierarchical controller does not run smoothly on the ground, potentially struggling with the stick-slip of the wheel-ground interactions. From this it seems that using active inference for the low-level, as compared to something like PID, is less suitable. However, that conclusion is too premature for this work as the available encoder feedback frequency for the active inference is way lower than what the MCU-level PID has access to, making the comparison less fair. The main contribution of this work lies in the insight in the implementation structure of hierachical active inference for continuous state-space control, which can be further built upon, using the lessons learned during implementation.