Skid-steer mobile robots present a unique challenge for reinforcement learning (RL) due to their nonholonomic constraints, dynamic wheel coupling, and high susceptibility to slip. Standard RL methods often struggle to achieve stable torque control in such settings, particularly under long-horizon tasks with sparse or delayed rewards. This thesis introduces a knowledge-assisted RL framework that systematically integrates expert demonstrations and curriculum learning to improve sample efficiency, training stability, and final policy performance.

The foundation of the method is a torque-based Deep Deterministic Policy Gradient (DDPG) agent, augmented through two key innovations: (1) KAMMA (Knowledge-Assisted Mixed Mode Actioning), a probabilistic switching mechanism that alternates between expert and learned actions to avoid interference artifacts and accelerate early-stage convergence; and (2) Curriculum-Driven System Identification, where the task complexity is gradually increased via staged velocity profiles to reveal underlying terrain-robot dynamics in a structured manner.

Experiments conducted in Isaac Sim demonstrate that this integrated KAMMA + Curriculum approach outperforms both baseline KA-DDPG and imitation-only variants across key metrics, including trajectory tracking error, policy smoothness, and convergence speed. The results confirm that combining staged learning with adaptive knowledge infusion enables robust torque-level control and offers a scalable template for learning-driven system identification in robotics. ... Read more

The autonomy of mobile robots has been greatly improved in recent decades. For these robots, the field of search and rescue is of particular interest. This thesis introduces a new method to let a mobile robot (Spot by Boston Dynamics) explore and search for victims in unknown environments. Existing methods include coverage, which aims to fully cover the environment as efficiently as possible. Exploration methods place more emphasis on gaining knowledge of the environment quickly, but do not actively search for victims. A new method based on active inference is introduced with the aim of combining exploration and exploitation behaviour within one framework. The active inference model is based on a graph representation of the environment, formulated as a POMDP. The framework is built up by incrementally more difficult cases. The first case allows a mobile robot to navigate a known graph to search for victims. The second case assumes a partially unknown graph. Uncertainty about the existence of unvisited nodes is included in the predictions. The final case adjusts the model to the agent's point-cloud- and camera sensors. The framework is then used in a simulation environment, showing how it can be implemented in real-world scenarios. To do so, the active inference framework is combined with techniques from information gain exploration. This thesis shows that active inference can be used in large unknown environments to carry out search and exploration. ... Read more

Significant work has been done in the field of computer vision focusing on learning and clustering methods. The use of improved learning methods has paved a way forward for researches to explore various theories to improve existing methods. One among various learning methods is Hierarchical learning which has showed impressive benefits and performance over traditional sequential learning approaches. In general, machine learning models require a lot of data for every new scenario which is not always possible and if so, is very expensive. Transfer learning, which focuses on transferring knowledge across trained machine learning models, is a promising machine learning methodology for solving the above problem. In this thesis, we propose an end-to-end neural network architecture on the NM500 neuromorphic chip using an incremental hierarchical learning approach. We first design a hierarchical representation of a taxonomy, develop a batch of pre-classifiers and use their output to construct a custom feature vector that is the input to the front-end network which learns the taxonomy. In other words, the taxonomy is embedded in the clustering method and not trained by a backpropagation algorithm. The custom feature vector has been structured to accurately incorporate the taxonomy based on the Manhattan distance norm. The structure has been proven mathematically and validated using experiments. A Radial Basis Function (RBF) is used for learning and a combination of RBF and K-Nearest Neighbors (KNN) for classification. The applicability of the proposed framework has been demonstrated on a road sign classification problem which is represented as a taxonomy. The ability of the framework to incrementally learn new categories and update the taxonomy online has also been shown. Lastly, we show a case of transfer learning where the entire back- end networks is used as a starting point to learn new features without significantly forgetting prior knowledge. This transfer learning framework showed comparable performance to the standard learning method in terms of accuracy while using significantly less labelled data. This work paves a way forward for researchers to develop transfer learning frameworks and more importantly explore neuromorphic hardware for machine learning tasks. ... Read more

Graph Neural Networks are a unique type of Deep Learning models that have a capability to exploit an explicitly stated structure of data representation. By design they carry a strong relational inductive bias, which is a set of assumptions that makes the algorithm prioritize some solutions over another, independent of observed data. This makes the method especially interesting for applications to problems, that are naturally relation-centric, or in which local interactions between features are the main value of interest. The presented research case, aims to explore GNN potential in application to an Ocean Acoustics problem. Using the geometric ray-tracing algorithm, BELLHOP, a large number of underwater sound propagation scenarios was simulated. Each scenario is described by a limited set of parameters and a Sound Speed Profile function. The latter, acting as a guideline for estimating paths of rays travelling through a water column, has a critical impact on sound propagation mode. For the data-driven model to effectively capture the acoustic phenomena, requires a mean of representing interactions in very scarce feature space and especially with respect to the nonlinear function representation of the sound speed. First, the solution of the problem is approached with a traditional Machine Learning model, a decision-tree algorithm XGBoost. In effect, some important characteristics of the collected data sample are revealed. Moreover, by testing inference capacity of the database with a reliable algorithm, gives an estimate of the properties of the Sound Speed Profile that have the biggest impact on sound propagation. It is proven, that with carefully engineered features, that include a degree of added expert knowledge, a standard model can achieve good accuracy of prediction. Secondly, a Knowledge Graph is designed to represent the whole context of explicitly stated expert knowledge, using concepts from Hydroacoustics. They are encoded in a form of relational structure connecting actual features of the data into logical categories. In this representation it can be used by the Knowledge Graph Convolutional Network model designed for the problem. A range of tests performed on KGCN proves that using a Graph Neural Network can be feasible to solve the problem, however it also reveals a range of issues regarding model's capability to handle the complexity of problem statement. ... Read more

Substantial efforts are being made to make robots more reliable and safe to work around humans. Robots often perform flawless demos in a controlled environment under the supervision of an operator but tend to fail in the real world when deployed for a long period of time due to faults and environmental disturbances. A robotic system is composed of different physical and software components whose characteristics are likely to change over time. Assumptions made about the system during the design phase may change over time, especially when a system is deployed for long periods. Such changes that are often ignored, need to be considered. Environments in which a robot operates are dynamic with high uncertainty and unpredictability. In such scenarios, capabilities such as situational awareness and self-adaptation will be useful to create more robust, resilient and reliable solutions. The objective for this thesis work is to develop a framework which will embed capabilities such as situational-awareness, context-awareness and self-adaptation within a robot. This research provides a novel, reusable and generalised localisation framework called Situation-Aware Self-Adaptive (SASA) localisation framework for robotics application. This framework is developed using knowledge representation and reasoning which will provide a robot with the capability of adapting according to the situation. We have demonstrated the applicability of the SASA framework to a mobile robot localisation use case. In this research work, we have demonstrated the performance of the framework during environmental disturbances due to poor illumination and featureless environment and internal fault due to component failure. We have also demonstrated the reusability, changeability and the consistency of SASA framework. This work showed that the situational-awareness and self-adaptation capability enhances the robot’s localisation ability and provides reliable localisation even in the case of environmental uncertainties and internal faults where conventional localisation systems fail. This thesis represents a leap forward in the direction of creating more reliable and resilient solutions for robotic applications and it lays the foundations for further research in this direction. ... Read more

Autonomous Underwater Vehicles (AUVs) are unmanned vehicles that give the opportunity to carry out lengthy and dangerous tasks autonomously. This is particularly useful for survey tasks, where the objective is to search the seafloor for objects. In this thesis work a planning system is developed that can plan a path for survey tasks, while considering environmental challenges such as communication limitations and location uncertainty. To compensate for location uncertainty, the planning system requires a higher level of abstraction compared to conventional path planning algorithms. For that reason, the planning problem is modelled in the PDDL, creating a powerful and flexible planning system which deals with the complex survey problem. Besides that, some additional planners are added to support the PDDL-planner and provide suitable plans for the AUV to carry out. The resulting plans are evaluated by simulation, showing that the planning system can successfully survey different scenarios. Besides that, the PDDL model is validated by means of the Event-B formal method, in order to obtain mathematical proofs of the validity of the planning model. The results are a step forward in achieving full autonomy of the AUVs. Besides that, a demonstration of the applicability of PDDL in real-world problems is given. ... Read more

Autonomous Underwater Vehicles (AUVs) are unmanned vehicles that are often used for searching an area of the seabed for objects, such as naval mines. For autonomous planning of such search operations, it is useful to be able to infer what tasks an AUV can perform and how well it can do so. The objective for this final thesis project therefore was to develop and implement a semantic knowledge model in the form of an ontology for assessing the capabilities of AUVs that perform mine search operations. For developing an ontology, first the use case and existing ontologies for related topics were studied to obtain an overview of the concepts and relationships that are most relevant for AUVs that perform mine search operations. The ontology was then developed by formalising these concepts and relationships, and by identifying instances of these concepts that are specific to the use case. By implementing this ontology and developing a reasoning process for it, automatic inference of the capabilities of an AUV based on the components it is equipped with was realised. In addition, a data-driven methodology for performance assessment was developed and incorporated in the ontology to provide a basis for automatic inference of the expected performance of an AUV. This performance assessment methodology was tested by performing simulations with an AUV in a virtual underwater environment. These simulations showed that the developed methodology can successfully be used to predict performance under different conditions. ... Read more