Active inference is a method for state estimation and control actions that is based on the Free Energy principle, which explains how biological agents infer the state of their environment and act upon it by maintaining a model of that environment and evaluating predictions. This method merges both action and sensory processing and is therefore a promising approach for robotic control. In active inference, the internal representations (or states) in the agent can be described by applying hierarchical dynamical models (HDMs). This offers a way to describe the dynamical development of states, and to structure them hierarchically. Despite the possibilities of this hierarchical approach, there are very few examples of hierarchical active inference being applied to robotics or control problems. This thesis aims to make a step in exploring the possibilities of active inference under a HDM. We describe the implementation of a control algorithm based on the hierarchical formulation of active inference on several continuous control problems, most importantly a 2-D robot arm. The algorithm is first demonstrated with a basic single level cart simulation, analysing the effect of the various parameters and inclusion of higher dynamic orders on the stability. A following simulation of multiple carts demonstrates a simple example of a hierarchical division of goals, and how high level objectives are realized by reaching lower level goals. It also shows how insolvable prediction errors at lower levels are propagated up the hierarchy. Finally, the simulation of a 2-D robot arm shows how these hierarchies can introduce goal-directed behaviour in practical control problems. We made several attempts to design a hierarchical generative model capable of realizing position-reaching behaviour in the robot arm, dictating joint angles from desired positions, with varying levels of success. Several pitfalls were encountered in choosing a suitable model for the control task. Most importantly, it was found that the convergence was faulty when higher levels contained more independent states. This emphasizes the information-reducing role of hierarchies. Lastly, we demonstrate that the hierarchical generative model is capable of adding complexity to the robots behaviour, by expanding the goal-reaching objective with a path-tracing task. In short, hierarchical active inference can be applied effectively to the demonstrated goal-reaching control problems, however this requires a careful consideration of the generative model for the task at hand.