In the field of robotics, consider the following problem scenario: In a robot environment, a simple robot must push objects to reference places while figuring out which objects can be pushed, what the best manipulation strategy is, or which objects are static and cannot be pushed. The problem scenario can be decomposed into three research topics which individually have received much attention from the research community; learning object dynamics [8, 37], Navigation Among Movable Objects (NAMO) [7, 13, 21, 23] and nonprehensile pushing [2, 5, 25, 41, 42, 43]. A combination of these three topics could lead to improvements in planning, execution time, and reasoning, but it has not been explored in the literature. This thesis proposes a robot framework that combines these three research topics. This framework comprises of three key components: the hypothesis algorithm, the hypothesis graph, and the knowledge graph. The hypothesis algorithm computes a hypothesis on how to relocate an object to a new pose by computing possible action sequences given certain robot skills. In doing so, the hypothesis algorithm creates an hypothesis graph that encapsulates the structure of the action sequences and ensures the robot eventually halts. Once a hypothesis is carried out on the robot, information about the execution, such as the outcome, the prediction error, the type of controller used and other metrics, are stored in the knowledge graph. The knowledge graph is populated over time, allowing the robot to learn, for instance, object properties and then refine the hypothesis computed to increase task performance, such as success rate and execution time. A new planning algorithm is proposed that can detect a when a path is blocked by an object, the hypothesis algorithm relies on the newly proposed planner to generate action sequences and to free blocked paths. This planner extends the double tree optimised Rapidly-exploring Random Tree algorithm [7]. The planner constructs a configuration space for an object and is provided with starting- and target pose for that object. The planner then converts these poses to points in configuration space to then search for a path connecting the starting configuration to the target configuration. For the new planner, objects are initially classified as “unknown” and can later be categorized as either “movable” or “unmovable”. The object type information is then used when constructing the configuration space for the newly proposed planning algorithm. Configuration space consists of the conventional free- and unmovable- (or obstacle) space and the newly proposed unknown- and movable space. To carry out the investigation, a mobile robot in a robot environment with movable and unmovable objects is created. The robot is given a task that involves relocating a subset of the objects in the robot environment through driving and nonprehensile pushing. The task can be broken down into individual subtasks that consist of an object and a target pose. Planning for a push or drive action occurs with the newly proposed planning algorithm that, if successful, completes a given task and populates the knowledge graph with learned object information. Information that can be used to determine which objects to manipulate and what strategy performs best to manipulate a specific object. In an effort to develop a robot framework that combines these three topics, a framework is created that shows improved task execution as a result to experience gained in the robot environment. The proposed framework performs equivalent or better compared to the state-of-the-art frameworks that are specialized in only two out of three research topics [47]. It can be concluded that the framework partly combines the three topics because learning system models with a system identification module is moved to the future work section. Instead, the proposed method selects the best available control and system model combination in the set of available control and system model combinations.

This thesis is a contribution to the research on Active Inference for Robotics. Active Inference is an intricate, intriguing theory from neuroscience, a field in which it has already gained a greater following and popularity. This theory, based on the underlying Free Energy Principle, provides a unified account of perception, action and learning in the biological brain. It has great explanatory power of the function of the biological brain and furthermore it is mathematically well-defined. This property makes the theory suitable for a translation to robotics, in which it can also provide a unified account of action and perception. This is not only elegant, but potentially very powerful too. The research for Active Inference in robotics is young, but the current research already shows that Active Inference indeed has great potential for robotics control. Literature on Active Inference is narrow and complex, and provides a lot of concepts to work with in a translation to robotics control. Once such concept are the generalised coordinates of motion, which are the instantaneous derivatives of a dynamic variable. The incorporation of generalised coordinates, especially in combination with the assumption that the noise encountered in a dynamic environment is coloured, has great potential to be beneficial for both action and perception when it comes to robot control in real environments. Generalised coordinates provide a reference frame for the gradient descent that is applied to provide the action and perception laws, which in a dynamic setting has to `hit a moving target'. Furthermore, in combination with coloured noise the generalised coordinates are advantageous for dealing with such noise. In this thesis, detailed research is provided with regards to the application of generalised coordinates in Active Inference for robotics. Current research for robotics in which Active Inference has been applied doesn't exploit the full potential of generalised coordinates. Therefore, this research aims to explore the constructs necessary to apply generalised coordinates of motion in an on-line Active Inference control loop of an LTI State Space system. A detailed derivation of the generalised precision, which relates generalised coordinates and coloured noise, is provided. A method for obtaining generalised output by means of finite differences is proposed, that constructs generalised coordinates from the on-line data in scenarios in which the environment does not provide the required generalised coordinates naturally. The method is implemented in the simulation of a one degree of freedom SISO LTI State Space scenario which highlights the potential but also the difficulties still faced when applying Active Inference for on-line robotics control. Besides the detailed derivations of some aspects of Active Inference for robotics, open problems are identified and suggested for future research that can potentially yield methods to apply Active Inference in robotics at full capacity, providing a true biologically plausible robot control method.

Autonomous robots are often successfully deployed in controlled environments. Operation in uncontrolled situations remains challenging; it is hypothesized that the detection of abstract discrete states (ADS) can improve operation in these circumstances. ADS are high-level system states that are not directly detectable and influence system dynamics. An example of a typical ADS problem that is used in this thesis is that of a wheeled robot driving through puddles of mud that, when entered, alters the velocity of the robot. When the robot is in such a puddle, it is in an ADS 'mud', and when it is not, it is in an ADS 'free'. ADS can be indirectly inferred through the analysis of lower-level data such as the velocity of the robot. The goal of this thesis is to design a general abstract discrete state estimator (ADSE) operating with limited prior knowledge. An ADSE is a hierarchical system for detecting changes in ADS. The ADSE should be general; applicable to multiple ADSE problems. The ADSE should further operate under limited prior knowledge: only assuming that the amount of ADS and the ADS that describes the regular operation are known. The basis for the ADSE designed in this thesis is a Gaussian hidden Markov model (GHMM), a hidden Markov model enhanced with Gaussian emissions. Randomly generated experiments are done on a simple but general ADSE problem. Two unsupervised learning methods derived from Expectation Maximization are evaluated, namely Baum-Welch (BW) and forward extraction (FWE). FWE is introduced in this thesis and is a simpler implementation of Viterbi extraction, leveraging assumptions of ADSE to in theory gain computational efficiency. We found that both BW and FWE exhibit superior performance compared to a likelihood-based baseline estimator when the maximum score of the learning curve is considered. When the final score is considered, in some cases, FWE displays a deteriorating learning curve, resulting in worse final scores compared to the baseline. Furthermore, it was found that the lower the overlap coefficient (therefore the less similar the ADS), the higher the maximum reached score. It was further shown that BW exhibits better convergence than FWE to the true model parameters. Besides this, FWE obtained comparable or in some cases even superior scores compared to BW. In general, from the results, the diversity of the experiments conducted, and the assumptions made we can conclude that the GHMM can be a general method for an ADSE with limited prior knowledge. To quantify the suitability of the GHMM for ADSE, further research should include the evaluation of different ADSE methods on the same problem. There exists a tradeoff between the lower computational cost FWE and the more stable but more computationally intensive BW learning. Therefore, future research can include a combination of these methods. Other extensions include extending the GHMM to a Gaussian mixture hidden Markov model to allow for the modeling of more complex distributions, or the application to multiple states or a changing environment.

The autonomy of mobile robots has been greatly improved in recent decades. For these robots, the field of search and rescue is of particular interest. This thesis introduces a new method to let a mobile robot (Spot by Boston Dynamics) explore and search for victims in unknown environments. Existing methods include coverage, which aims to fully cover the environment as efficiently as possible. Exploration methods place more emphasis on gaining knowledge of the environment quickly, but do not actively search for victims. A new method based on active inference is introduced with the aim of combining exploration and exploitation behaviour within one framework. The active inference model is based on a graph representation of the environment, formulated as a POMDP. The framework is built up by incrementally more difficult cases. The first case allows a mobile robot to navigate a known graph to search for victims. The second case assumes a partially unknown graph. Uncertainty about the existence of unvisited nodes is included in the predictions. The final case adjusts the model to the agent's point-cloud- and camera sensors. The framework is then used in a simulation environment, showing how it can be implemented in real-world scenarios. To do so, the active inference framework is combined with techniques from information gain exploration. This thesis shows that active inference can be used in large unknown environments to carry out search and exploration.

The problem of assisting users in comprehending the robotic scenario information in a retail setting has been studied. To design the system, an integrated ontology composed of several IEEE standard ontologies and a labelled property graph (LPG)-based ontology modified from the Web Ontology Language (OWL)-based ontology was proposed to symbolize information in the robotic environment. Then, a knowledge graph (KG)-based chatbot was developed to provide natural language interaction with users. A case study in a retail setting was designed, and the results were analyzed. The effectiveness of our designed system has been experimentally validated in both static and dynamic scenarios, with at least 1.5 times improvements.

Task and Motion Planning (TAMP) has progressed significantly in solving intricate manipulation tasks in recent years, but the robust execution of these plans remains less touched. Particularly, generalizing to diverse geometric scenarios is still challenging during execution. In this work, we propose a reactive TAMP method to deal with disturbances and geometric ambiguities by combining an active inference planner (AIP) for online action selection with a proposed multi-modal model predictive path integral controller (M3P2I) for low-level control. The AIP generates online alternative plans, each of which is translated into a cost function to be sampled for the proposed method. The proposed M3P2I then uses a parallelizable physics simulator for throwing different rollouts, leading to a coherent optimal solution by averaging the weighted samples based on their costs. Our method empowers real-time adaptation of action sequences to rectify failed plans, while also computing low-level motions to address dynamic obstacles or disturbances that could potentially invalidate the existing plan. Theoretical findings are validated in simulation and in the real world. We show that the framework exhibits reactiveness in different scenarios, including battery charging, push-pull among obstacles, and pick-place with disturbances. We show that our framework outperforms an off-the-shelf RL method in the reactive pick-place task in terms of position error and orientation error. We also show that M3P2I is generalizable in combining different constraints, such as generating hybrid motions of push and pull for the mobile robot, and grasping objects with different grasping poses. The real-world experiments show that the system exhibits reactiveness and robustness against human disturbances in a variety of manipulation tasks. The supporting videos can be found at https://sites.google.com/view/m3p2i-aip.

Achieving human-like action planning requires profound reasoning and context-awareness capabilities. It is especially true for autonomous robotic mobile manipulation in dynamic environments. In the case of component failure, the autonomous robotic system requires reliable adaptation capabilities combined with a consistent understanding of the task, environment, and the robot’s capabilities for successful task completion. Recent research has shown that Active Inference, a unifying neuroscientific theory of the brain, has the potential to intrinsically handle substantial uncertainties in the system, resembling the adaptability of humans. These works, however, have the following limitations: (1) no distinction is made between actions with some commonality, capable of satisfying similar tasks, and (2) actions are assumed to be always feasible when preconditions are satisfied, regardless of their context. Given the situation, certain actions satisfying a task might not lead to task succession. This work proposes the AI for retail (Airet) framework, a novel extension of action planning through Active Inference for mobile manipulation. The Airet framework uses Bayesian networks and Ontological Reasoning to facilitate context-awareness in action planning through Active Inference. Reasoning on robot components, action-, manipulation- and environmental constraints is facilitated through a description-logic-based reasoner and an OWL-based ontology containing concepts relevant for action selection in a retail context. The capabilities of the Airet framework are demonstrated through the following cases (1) irrecoverable task & component. Failure prevention when dealing with ill-defined tasks, (2) Selection of the best action given the situation & the component capabilities through context-awareness (3) failure recovery & adaptation when dealing with component failure. Lastly, these situations are compared with research on reactive task planning through Active Inference without context-awareness. This thesis represents a leap forward from the current state-of-the-art in Active Inference for task planning in robotics, laying the foundations for further research in the direction of this thesis

This thesis proposes the novel Behaviour Tree Update Framework (BTUF) for the initial construction and continuous incremental adaptation of Behaviour Trees (BTs) for applications in Learning from Demonstration (LfD) frameworks to create complex robot behaviours associated with Activities of Daily Living (ADL) without requiring the user to have a programming or engineering background. BTUF implements several methods towards that end. Automatic generation of the fundamental structure of BTs within BTUF allows for easy human operation of the framework's Text-based User Interface (TUI). Saving and loading of constructed trees facilitates easy expansion and reusability of constructed trees. By expanding upon an initial base behaviour, seemingly simple behaviours can be adapted to facilitate novel instances thereupon, increasing the complexity and functionality of the constructed tree over time. Experimental validation in the form of a user study has provided proof-of-concept within simulation and has given insight in the initial overall performance and general system acceptance of BTUF. Future work is recommended for further validation and improvement of the proposed framework to one day realise a real-life application within healthcare robotics.