Unmanned Aerial Manipulators (UAMs) have revolutionized industrial infrastructure inspection. But when a drone deploys an extended manipulator arm to carry dense Non-Destructive Testing (NDT) sensors, the payload severely shifts the system's Center of Gravity (CoG). This forward shift generates static pitch torques that can quickly overwhelm and saturate standard flight controllers.

To resolve this, this thesis presents the design, digital simulation, and experimental flight validation of a novel aerial manipulator. The architecture features a 1-Degree-of-Freedom (1-DOF) parallel linkage equipped with auxiliary tip motors. This system utilizes an Active Moment Compensation (AMC) strategy, in which the tip rotors generate dedicated vertical lift to continuously counteract the payload's gravitational torque, fundamentally decoupling the manipulator's reach from the primary quadrotor's stabilization limits. Before physical deployment, the control logic was validated in a high-fidelity Software-in-the-Loop (SITL) digital twin using ROS 2 and Gazebo Harmonic. Crucially, the PX4 Control Allocator was mathematically extended from a standard 4-rotor matrix to a custom 6-rotor geometry, enabling the autopilot to natively map thrust across the system.

Extensive parametric flight testing proved the core kinematic hypothesis: because the parallel linkage forces the tip-rotors to remain perfectly horizontal, the required compensation thrust is a mathematical constant, completely independent of the arm's pitch angle. Experimental data confirmed that the AMC architecture neutralizes extreme CoG shifts while preserving the dynamic thrust headroom required for stable flight. Additionally, vibration and telemetry analyses demonstrated that elevating the mechanism isolates it from the primary rotor downwash, reducing structural resonance and altering attitude control performance. By successfully de-risking this active-thrust architecture, this research provides a rigorously verified, scalable framework for deploying heavy industrial sensors at extended standoffs. ... Read more

The need for international transport of goods has increased for decennia, due to the globalisation of supply chains. The impact on the environment has gotten increased attention in the last few years, with different initiatives aiming to decrease transport emissions. While the EU has released a white paper stressing the need for modal shift towards rail transport, clear results remain absent. Therefore, this thesis explores improving the efficiency of the railway transport in the port area, specifically the port of Rotterdam. The goal is to both improve operational efficiency to decrease direct emissions, and incentivise modal shift towards railway transport.

To achieve these gains, an existing mathematical model is adapted to optimise the schedule for a single railway feeder services operator. A multi-objective function is minimised, to combine orders, reduce locomotive use and improve on time delivery. The model is benchmarked against a greedy algorithm, and structurally outperforms it.

Next, the feeder train services model (FTSM) is then used to investigate cooperative scheduling approaches. Cooperative game theory is used and the FTSM is run with stand-alone and pooled railway operators. For all scenarios tested, cooperating yields benefits, with cost reductions ranging from 25\% to 58\%, compared to stand-alone operation. The stable coalitions presented by this thesis present further gains in network capacity, as the pooled operators occupy less tracks.

This thesis fills the gap of port-specific railway freight transport, for which it both presents a novel mathematical model, and a cooperative strategy. The scenarios tested show benefits for all stakeholders, providing a solid base for further research and implementation. ... Read more

Conventional post-stroke gait rehabilitation is labour-intensive, often requiring multiple therapists for physical assistance. Both robotic and aquatic therapies have been proposed to address this: robots can reduce therapist workload and provide adaptable feedback, while aquatic environments offer apparent weight reduction, improved balance, and muscle strengthening. Virtual aquatic therapy combines both approaches by haptically rendering the fluid dynamics of virtual water. In addition to the sagittal degrees of freedom offered by existing implementations, the treadmill-based exoskeleton used in this study enables hip abduction and lateral pelvis movements. A physics-based fluid rendering model is developed to compute drag and buoyancy forces acting on virtual legs moving through fluids. To match the real-world setup, the model simulates treadmill walking rather than the conventional free walking and includes the additional axes of motion. Simulations confirm that the model's outputs are consistent with the literature and scale properly with movement speed, submersion depth and the type of fluid being rendered. Through a proof-of-concept experiment, the successful integration into the exoskeleton’s control framework is validated, demonstrating the feasibility of rendering diverse fluid environments with this system. ... Read more

Background: Spinal cord injury (SCI) disrupts brain–peripheral nervous system communication, causing total or partial loss of motor, sensory, and autonomic functions below the lesion. Although functional electrical stimulation (FES) cycling rehabilitation can improve cardiopulmonary function, muscle mass, and locomotion, its clinical adoption is hindered by complex parameter tuning and a lack of standardized and personalized protocols.

Objective: The goal of this work is to first develop a methodology for the creation of FES patterns from electromyography (EMG), specifically from muscle synergy analysis. Then, to study how the obtained patterns compare to a standard one. We hypothesized that (1) the standard FES pattern would diverge from healthy muscle-activation timing, (2) it was designed for intermediate cadences, and (3) synergy-based patterns would yield smoother pedaling and better user ratings.

Methods: During data acquisition, we recorded EMG of eight leg muscles from eight healthy controls and two SCI patients while cycling at 30, 50, and 70 revolutions per minute (rpm) on a recumbent ergometer. Non-negative matrix factorization extracted muscle synergies and reconstructed EMG envelopes, which defined personalized stimulation ranges and amplitude profiles for each muscle. We quantified inter-limb symmetry and deviations of the personalized pattern from the standard one to characterize how each FES parameter set differed across trials. In the evaluation phase, eight healthy participants tested three patterns in random order for six minutes: the standard pattern, a fixed-50-rpm synergy-based pattern, and a synergy-based pattern adapting to the real-time cadence. Outcome measures were rpm variability and user perception ratings.

Results: Two of our three hypotheses were confirmed: (1) the standard FES pattern diverged significantly from healthy muscle-activation timing, and (2) it appeared to have been indeed designed for mid-range cadences (50 rpm approx.). However, the synergy-based personalization did not yield clear improvements over the standard pattern, and subjective preferences did not consistently align with objective metrics. Since the statistical tests yielded no significant difference between trials, two illustrative cases were shown, highlighting variability.

Conclusion: A practical workflow is recommended to first generate candidate patterns from muscle synergy analysis, conduct brief comparative trials using application-specific measures and user feedback, and then select the pattern that best balances these criteria for the intended rehabilitation protocol. ... Read more

Recent research suggests that haptic feedback—the use of physical stimuli to simulate tactile experiences—plays a crucial role in simulations in virtual reality (VR), as it can enhance immersion and facilitate motor learning. Unlike real-world objects, virtual objects lack the property of mass, necessitating its simulation through haptic devices to convey a realistic sense of weight. However, rendering weight remains a challenge, particularly for ungrounded haptic devices, which maintain a free range of motion but often face limitations such as high latency, side effects through noise, vibrations, and airflow, or the need for expensive equipment. In this work, we present LeVR, a low-cost, portable haptic proxy that simulates weight by rendering the vertical forces experienced when lifting objects—within the system’s constraints—by leveraging the force generated by an accelerated mass. The system comprises a linear rail and a capstan drive mechanism and ncorporates an impedance-based control scheme. We characterized the system’s response through step and frequency analyses. Results show that LeVR can produce a force output within a latency of 2.5 ms and render forces at frequencies ranging from 1 to 11 Hz. Furthermore, we conducted a pilot user study in which participants sorted five virtual objects by weight, ranging from 17 g to 227 g, solely based on the stimuli produced by our prototype. The results indicate that participants could generally distinguish between different stimuli, though limitations such as force instability, oscillations, and fatigue affected sorting accuracy. With our proposed system we aim to contribute to research on weight perception, to ultimately increase the effectiveness of skill acquisition and motor learning in VR. ... Read more

Traditional path-planning methods for mobile robots typically focus on avoiding obstacles but often fall short when obstacles block the path to the goal. This paper addresses the challenge of Navigation Among Movable Obstacles (NAMO), where a single robot can reposition obstacles to create previously inaccessible pathways. We introduce SVG-MPPI, a novel approach that integrates semantics to incorporate continuous movability into both path planning and local control strategies, allowing the robot to navigate cluttered environments by moving obstacles as needed to reach its goal.

SVG-MPPI refines the traditional Visibility Graph (VG) method by introducing the Semantic Visibility Graph (SVG). In this advanced approach, additional nodes are placed near movable obstacles, with the movability of these obstacles assessed based on their estimated mass—a key semantic property defining movability. This enables the modeling of potential passages through these barriers. The cost of push actions needed to navigate to these additional nodes is incorporated into the path-finding algorithm, eliminating the need for explicit obstacle placement and reducing the overhead of custom task planning. The local control strategy employs Model Predictive Path Integral (MPPI), which uses the IsaacGym physics engine to simulate robot and obstacle state transitions. MPPI minimizes trajectory contact forces as part of its objective, thereby reducing the push actions executed by the robot. The system supports optional replanning by continuously evaluating obstacle movability and adjusting the path if actual conditions deviate from initial estimates.

Our solution was evaluated through both qualitative and quantitative experiments. Qualitative experiments demonstrated the algorithm’s success in a simulated environment, where it effectively handled path execution and replanning scenarios. In real-world testing, an omnidirectional robot successfully pushed an obstacle and established a path to a goal. Quantitative analyses compared SVG-MPPI with its unaltered planning method (VG), which has no notion of movability, and Rapidly-exploring Random Tree (RRT) adapted to include binary movability. Each of these path planning methods was equipped with MPPI, which had a similar notion of movability as the planner. Results showed that SVG-MPPI consistently outperformed both VG and RRT in terms of path planning success rate and execution success rate. Furthermore, SVG-MPPI exhibited lower cumulative contact forces over the trajectory, indicating that the integration of continuous movability allowed the algorithm to select paths of least resistance and navigate around obstacles where possible.

Overall, SVG-MPPI represents a significant advancement in NAMO planning by offering a cohesive solution that addresses the complexities of navigating random, cluttered environments with movable obstacles. By prioritizing direct progress toward the goal while repositioning obstacles, SVG-MPPI introduces a novel, integrated approach with promising results and substantial potential for future research and development.
... Read more

Humans are our best example of the ability to learn a structure of the world through observation of environmental regularities. Specifically, humans can learn about different objects, different classes of objects, and different class-specific behaviors. Fundamental to these human abilities is evolved sensory hardware and automatic pattern recognition systems thought to be powered in part by the leading neuroscience theory of predictive coding. Artificial intelligence research is often inspired by neuroscience and algorithms already exist that implement predictive coding. In this paper, we seek to evaluate a leading predictive coding video-prediction algorithm, PredNet, for its ability to perform human-like learning of the types mentioned. By successfully training PredNet on a custom Simple Shape Motion (SSM) video dataset that explicitly requires structure learning to occur in order to accurately predict the next frame, we establish that PredNet is capable of rudimentary structure learning. We investigate PredNet filters and feature maps but find scant evidence of truly symbolic knowledge, and propose instead that PredNet performs semi-symbolic learning. We perform ablation studies that reveal the aspects of PredNet that critically contribute to its structure learning ability. Finally, we detail a set of modifications made to PredNet to allow object-centric processing as a promising step change towards human-like structure learning. Evaluation results and investigations are provided. Performance was slightly worse than Baseline, likely due to a noted implementation flaw. Code and instructions to reproduce dataset creation and model training / evaluation are available at https://github.com/ofSingularMind/parallel_prednet. ... Read more

This research investigates the benefits of using a trajectory tracking controller based on a dynamic model for a four-Mecanum-wheeled vehicle (FMWV) over a kinematic-model-based controller. An FMWV was designed and built, incorporating both hardware and software components. Two Linear Quadratic Regulators (LQRs) based on kinematic and dynamic models were implemented. The dynamic model includes friction estimation, while the kinematic model assumes a no-slip condition. Simulation results indicate that the dynamic model reduces overshoot and improves trajectory tracking in low-friction scenarios compared to the kinematic model. High-friction scenarios show comparable performance for both controllers. Experimental results align with simulations, though some deviations highlight areas for further improvement. Overall, the dynamic-model-based controller demonstrates superior performance in low-friction conditions, reducing translational root mean square error (RMSE) and maximum path deviation (MaxE). ... Read more

The rapidly growing volume of parcel shipments is straining transportation and logistics sectors, highlighting the need for innovative solutions to optimize packing and loading processes. The online bin packing problem (BPP), an NP-hard computational problem, finds practical applications in numerous sectors, including modern packaging and intelligent logistics. This study proposes a novel reinforcement learning (RL) approach to tackle the online 3D-BPP emphasizing applicability and versatility. The key innovation is the representation of the packing scene as a graph, enabling effective encoding of task-specific high-level features. This graph-based structure serves as the foundation for an RL agent designed to learn an optimal packing strategy through dynamic interaction with the environment. The proposed approach uniquely operates within the continuous domain, enhancing generalization across diverse packing tasks. Experimental evaluations in both simulated environments and a real-world setting demonstrate that the solution achieves state-of-the-art performance across multiple complex three-dimensional packing scenarios. ... Read more

Human Theory of Mind (ToM), the ability to infer others’ mental states, is essential for effective social interaction. It allows us to predict behavior and make decisions accordingly.In Human Robot Interaction (HRI), however, this remains a significant challenge, especially in dynamic, real-world scenarios. Enabling robots to possess ToM-like capabilities has the potential to greatly improve their interaction with humans. Recent advancements have introduced Large Language Models (LLMs) as robot controllers, leveraging their strengths in generalization, reasoning, and code comprehension. Some have claimed that LLMs may exhibit emergent ToM capabilities, but these claims have yet to be substantiated with rigorous evidence. This study investigates the ToM-like abilities of Multimodal Large Language Models (MLLMs) by creating a benchmark dataset from humans performing object rearrangement tasks in a simulated environment. The dataset visually captures the participants’ behavior and textually captures their internal monologues. Based on this dataset (text, video, or hybrid) three state-of-the-art models made predictions about the participants’ belief updates. While the results do not conclusively establish ToM capabilities in MLLMs, they offer promising insights into mental model inference and suggest future directions for research in this domain. ... Read more

Dynamic obstacle avoidance remains a crucial research area for autonomous systems, such as Micro Aerial Vehicles (MAVs) and service robots.
Efforts to develop dynamic collision avoidance techniques in unknown environments have proliferated in recent years. While these methods exhibit impressive and reliable performance in simpler environments, their efficacy in more challenging settings remains an area ripe for enhancement. The difficulty of these environments arises from a multitude of factors, and currently, no standardized approach exists to quantify this complexity. Additionally, to fairly compare different dynamic collision avoidance strategies, it's essential to assess them in environments with a similar degree of difficulty. Therefore, devising a metric capable of accurately gauging the intricacy of dynamic environments becomes imperative.


Building on this context, this master's thesis endeavors to fill this critical gap through three contributions: 1) The establishment and validation of map difficulty metrics that represent the difficulty of dynamic environments, 2) The introduction of a robust benchmarking pipeline to critically validate the representativeness of the proposed metrics and evaluate various collision avoidance strategies, and 3) The provision of a framework for comparative analysis of different planning strategies, utilizing the introduced map difficulty metric.

The proposed survivability metric effectively captures environmental complexity. Its validity is evidenced by a notable correlation with the success rates of typical collision avoidance methods, with over 1.7 million collision avoidance trials on over six hundred maps, securing a Spearman's Rank correlation coefficient (SRCC) of over 0.9. This metric serves as an indispensable tool for facilitating fair comparisons in this dynamic research domain. More importantly, it offers valuable insights for the future refinement and improvement of dynamic collision avoidance strategies, making a contribution to the continuous advancement of autonomous systems. ... Read more

In the field of robotics, consider the following problem scenario: In a robot environment, a simple robot must push objects to reference places while figuring out which objects can be pushed, what the best manipulation strategy is, or which objects are static and cannot be pushed. The problem scenario can be decomposed into three research topics which individually have received much attention from the research community; learning object dynamics [8, 37], Navigation Among Movable Objects (NAMO) [7, 13, 21, 23] and nonprehensile pushing [2, 5, 25, 41, 42, 43]. A combination of these three topics could lead to improvements in planning, execution time, and reasoning, but it has not been explored in the literature. This thesis proposes a robot framework that combines these three research topics. This framework comprises of three key components: the hypothesis algorithm, the hypothesis graph, and the knowledge graph. The hypothesis algorithm computes a hypothesis on how to relocate an object to a new pose by computing possible action sequences given certain robot skills. In doing so, the hypothesis algorithm creates an hypothesis graph that encapsulates the structure of the action sequences and ensures the robot eventually halts. Once a hypothesis is carried out on the robot, information about the execution, such as the outcome, the prediction error, the type of controller used and other metrics, are stored in the knowledge graph. The knowledge graph is populated over time, allowing the robot to learn, for instance, object properties and then refine the hypothesis computed to increase task performance, such as success rate and execution time. A new planning algorithm is proposed that can detect a when a path is blocked by an object, the hypothesis algorithm relies on the newly proposed planner to generate action sequences and to free blocked paths. This planner extends the double tree optimised Rapidly-exploring Random Tree algorithm [7]. The planner constructs a configuration space for an object and is provided with starting- and target pose for that object. The planner then converts these poses to points in configuration space to then search for a path connecting the starting configuration to the target configuration. For the new planner, objects are initially classified as “unknown” and can later be categorized as either “movable” or “unmovable”. The object type information is then used when constructing the configuration space for the newly proposed planning algorithm. Configuration space consists of the conventional free- and unmovable- (or obstacle) space and the newly proposed unknown- and movable space. To carry out the investigation, a mobile robot in a robot environment with movable and unmovable objects is created. The robot is given a task that involves relocating a subset of the objects in the robot environment through driving and nonprehensile pushing. The task can be broken down into individual subtasks that consist of an object and a target pose. Planning for a push or drive action occurs with the newly proposed planning algorithm that, if successful, completes a given task and populates the knowledge graph with learned object information. Information that can be used to determine which objects to manipulate and what strategy performs best to manipulate a specific object. In an effort to develop a robot framework that combines these three topics, a framework is created that shows improved task execution as a result to experience gained in the robot environment. The proposed framework performs equivalent or better compared to the state-of-the-art frameworks that are specialized in only two out of three research topics [47]. It can be concluded that the framework partly combines the three topics because learning system models with a system identification module is moved to the future work section. Instead, the proposed method selects the best available control and system model combination in the set of available control and system model combinations. ... Read more

Machine learning can be effectively applied in control loops to robustly make optimal control decisions. There is increasing interest in using spiking neural networks (SNNs) as the apparatus for machine learning in control engineering, because SNNs can potentially offer high energy efficiency and new SNN-enabling neuromorphic hardwares are being rapidly developed. A defining character of control problems is that environmental reactions and delayed rewards must be considered. While reinforcement learning (RL) provides the fundamental mechanisms to address such problems, realizing these mechanisms in SNN learning has been underexplored. Previously, schemes of spike timing dependent plasticity (STDP) learning modulated by factors of temporal difference (TD-STDP) or reward (R-STDP) have been proposed for RL with SNN. Here we designed and implemented an SNN controller to explore and compare these two schemes by considering Cart-Pole balancing as a representative example. While the TD-based learning rules are very general ones, the resulted model exhibits rather slow convergence, producing noisy and imperfect results even after prolonged training. We show that by integrating the understanding of the dynamics of the environment into the reward function of R-STDP, a robust SNN-based controller can be learnt much more efficiently than by TD-STDP. The work of this master thesis project has also been published as a paper in Electronics, Vol. 12. ... Read more

Modern vehicles have multiple different buses to communicate between components, like CAN (FD) and FlexRay. ZF builds ”innovation vehicles” with new components to showcase and test them. These components are connected to the automotive buses. ZF uses a web-based Human Machine Interface (HMI) to control and view the state of these parts. This HMI is needed because some systems are not visible or controllable in regular operation. A gateway is required to connect the HMI and the buses. ZF currently uses a CAN to WebSocket gateway that does not support other buses. There is no readily available hardware with the required buses and interfaces. A WebSocket interface is required, as the HMI is running in a browser, limiting the possible protocols.
Therefore, the challenge for this thesis is how to (re)design this HMI system for future innovation vehicles with buses besides CAN. Each vehicle can have a different number and types of buses, so the system must be able to cope with this. The HMI system is not the only part in the network, so it also must be efficient to not interfere with other systems, like network cameras. This redesign was done by analysing the types of buses, the hardware that could be used, and the software components needed for this flexible system. The software components were mapped onto the available hardware to make the architecture as flexible and efficient as possible. Two mappings are proposed, one using a Software Gateway, a custom application with support for different hardware interface drivers, capable of running on Windows, Linux and in containers. The other mapping uses WebSockify, where WebSocket messages are converted to TCP/UDP messages.
These two mappings were combined into a single architecture to combine the features and possibilities of both systems. Then a Minimum Viable Product (MVP) was made to test the envisioned architecture, showing excellent results compared to the current solution while adding more flexibility and other features. Using this new HMI system, the HMI developers can interface with more types of networks, build HMIs that connect to multiple different vehicles and make distributed HMI systems. ... Read more

Humans and other biological bipedal walkers are extraordinarily agile and robust. This is especially apparent when certain features of the environment are unknown such as unexpected downsteps (for example, suddenly walking off the side-walk or not expecting the last stair when descending). Humans can negotiate these downsteps ---both expected and unexpected--- with remarkable agility and ease. The contributions from active compensation, passive dynamics, and reflexes that humans employ to overcome these downsteps are however not inherently present in bipedal robots. This motivates us to assess whether externally observed behavior can be used to achieve the same capabilities in bipedal robots. One of the challenges with this proposal is the morphological differences between humans and robots. Taking the bipedal robot Cassie as an example, these differences predominantly constitute to overall- and segment mass, leg morphology, and a lack of an upper body. The observed behavior from the human should subsequently be scaled to represent an appropriate nominal and compensatory behavior of the robot.

This thesis aims to systematically study the translation of this human behavior to bipedal walking robots, regardless of the morphology of the walkers. We start from human experimental data where nominal walking, expected downstep, and unexpected downstep trials were conducted. The human data is analyzed from the perspective of a reduced-order representation of the human. The reduced-order representation encodes the center of mass dynamics and contact forces. An equivalent reduced-order model is used to represent the bipedal robot, which allows for the translation of the nominal walking and downstep behaviors between human and robot. The morphological differences between the human and the robot are therefore resolved by realizing dynamically equivalent behavior which is embedded into the full-order dynamics of a bipedal robot via optimization-based controllers. The results demonstrate traversing expected and unexpected downsteps in simulation on the underactuated 3D walking robot Cassie. ... Read more

Recent developments have enabled the mass production of cheap, high-performance multirotors. As a result of this, the multirotor has found a large variety of different uses. Testing new algorithms using real-life flights is costly in terms of time and potentially in terms of materials in the case of a crash. Simulators have quickly gained prominence as a tool in algorithm development. The accuracy of these simulators is vital to ensure the step from simulated flights to real-life testing is as small as possible. Current simulator models are held back due to a lack of properly identified multirotor coefficients. Often, the coefficients from a different multirotor are used, even if the new multirotor is vastly different in shape and size. Additionally, most researchers do not have a sufficient overview of which parts of the simulator are most important in providing an accurate simulator environment.

This research will give an overview of the different parts that influence the simulator results. The simulation of the motor controllers and rotor aerodynamics is identified as a major contributor to inaccuracies. The coefficients used in these two components will be found by performing a system identification process of the NXP HoverGames quadcopter. This identification process entails determining the mass, dimensions, inertia, motor response and thrust, torque, rotor drag and rolling moment coefficients. Experiments will be performed to find the correct coefficient values and to evaluate the impact that each effect has on the simulator's performance. A focus is put on ease of repeatability, such that other researchers can use the same process for their specific multirotor.

The theoretical background behind each effect in the motor controller and rotor aerodynamics models is discussed. This theoretical knowledge is used to perform experiments to find the coefficients for each effect and validate them. As a result, new coefficients for the motor controller simulator, thrust, torque and rotor drag were found. Additionally, this process showed that the rolling moment is much smaller than the errors currently existing in the model and this effect has, therefore, been ignored. The motor controller simulation is identified as the largest cause of inaccuracies in the current model. This simulation maps the motor commands to the output rotational velocity of the rotor and simulates the time response of this system. The formulas currently used for this do not provide a sufficient model of real-life behaviour. An analysis is done to gain a better understanding of what variables influence motor behaviour.

Finally, the new coefficients are put to the test in a variety of different trajectories to compare the trajectory tracking performance with the old model. This comparison consists of two parts. In the first part, a qualitative analysis is used to understand the difference in simulator behaviour. In the second part, a set of metrics are defined to quantify the difference in accuracy. This comparison process is used to prove that the new model provides significant improvements compared to the old coefficients, especially in the case of faster, more aggressive manoeuvres. ... Read more

A common method to investigate multisensory integration is using multisensory illusions. The rubber hand illusion is one of the best-known multisensory illusion used in clinical applications. By stroking a visible rubber hand and the participant’s occluded hand, the illusion arises that the rubber hand belongs to the participant. Possible applications based on this illusion are for neurorehabilitation or developing robotic devices. These applications are still in a very initial state, and this thesis aims to take these possible applications a step further. The thesis provides the results of a rubber hand illusion experiment over time and a dynamic model related to the embodiment of the rubber hand. The dynamical model can visualize the proprioceptive drift that arises from the illusion and predict the time it takes to experience it. The results of the experiment and other empirical findings from the literature form the basis of the model. The experiment measured these factors and the feeling of ownership and agency over time. The results from this experiment have much variance but are in line with the literature. The dynamical model fails to visualize the body ownership and agency results. Adding extensions and improvements could make this dynamic model more complete and applicable in many research fields. ... Read more

An end-to-end framework is developed to discover physical laws directly from videos, which can help facilitate the study on robust prediction, system stability analysis and gain the physical insight of a dynamic process. In this work, a video information extraction module is proposed to detect and collect the pixel position of moving objects, which would be further transformed into physical states we care about. A physical law discovery module is developed to learn closed-form expressions based on the extracted physical information. The video information extraction module takes advantage of contour detection and Hough transformation to extract position information. The physical law discovery module includes a deep neural network-like hierarchical structure Mathematical Operation Network (MathONet) which is consisted of basic mathematical operations. We develop a sparse Bayesian learning algorithm to learn both the topology and parameters of dynamic systems. Several simulated videos were generated to illustrate Newton’s law of motion, the state space of a Duffing oscillator, and the pendulum motion equation. By demonstrating on these examples, our method can discover the corresponding governing function without requiring much prior information. ... Read more

Intelligent agents must pursue their goals in complex environments with partial information and often limited computational capacity. Reinforcement learning methods have achieved great success by creating agents that optimize engineered reward functions, but which often struggle to learn in sparse-reward environments, generally require many environmental interactions to perform well, and are typically computationally very expensive. Active inference is a model-based approach that directs agents to explore uncertain states while adhering to a prior model of their goal behaviour. This paper introduces an active inference agent which minimizes the novel free energy of the expected future. Our model is capable of solving sparse-reward problems with a very high sample efficiency due to its objective function, which encourages directed exploration of uncertain states. Moreover, our model is computationally very light and can operate in a fully online manner while achieving comparable performance to offline RL methods. We showcase the capabilities of our model by solving the mountain car problem, where we demonstrate its superior exploration properties and its robustness to observation noise, which in fact improves performance. We also introduce a novel method for approximating the prior model from the reward function, which simplifies the expression of complex objectives and improves performance over previous active inference approaches. ... Read more