Deep learning has led to strong performance in recognizing materials from touch signals, but it is often difficult to understand which parts of those signals drive a model’s decision. This thesis studies accurate and interpretable tactile material classification on the public SENS3 dataset using three dominant cue sources: thermal transients during static contact, deformation dynamics during pressing, and friction/vibration cues during sliding. An adjective-mediated pipeline that first predicts psychophysical rating distributions and then classifies materials is compared against a direct multimodal classifier that fuses modality-specific encoders. The direct multimodal model shows strong generalization, reaching 0.896 test accuracy across seven retained material classes. To better understand the learned decision process, Integrated Gradients was used as the main explanation method, combined with Temporal Saliency Rescaling for models with a single classification output. The resulting attribution maps were then summarized into contributions at the level of signal type, interaction phase or bin, and individual measurement channel. The resulting explanations reveal class-conditional cue usage aligned with interaction structure: transient thermal phases dominate for metal-like materials, pressing dynamics contribute strongly for compliant/textile-like classes, and sliding evidence concentrates in low-force regimes where friction and vibration cues are most informative. Overall, the results demonstrate that high-performing tactile material recognition can be combined with interpretable, physically grounded attribution summaries, improving trust in model decisions for haptic interfaces and embodied systems. ... Read more

Affordances, or action possibilities, have been explored to enable robotic manipulation with everyday objects, however the effect of an agent's embodiment has not received much attention. Here we investigate how embodiment changes affordances between a human and robot. We present a method to automatically generate affordance pseudo-labels from a robotic manipulator for the task of grounding (localising) affordances on an object, as there is no such existing dataset. We then propose a general model for embodiment-conditioned affordance grounding, and explore three ways to condition on the embodiment. Our model learns to perform an affine transformation on image embeddings based on the effect of embodiment on the affordance. We evaluate all three variants of our model and compare them to a variant without embodiment conditioning and a state-of-the-art affordance grounding method. The results show that our best performing model decreases affordance prediction error by 25% when compared to the variant without embodiment conditioning and by 68% when compared to the state-of-the-art method. Through our results we demonstrate that embodiment matters when perceiving affordances. ... Read more

This study investigates how font size, text-drawing style, and drop shadow affect the visibility, legibility, and comprehension of overlaid text. The purpose of this study is to better understand the role of these design factors and to offer practical guidelines for presenting information effectively on see-through displays. Twenty-five participants completed a visual search task, where they were asked to locate a target word presented under varying conditions on a complex background. A full factorial design was employed, incorporating four font sizes (0.10°, 0.15°, 0.20°, and 0.60°), six text-drawing styles (plain green text, plain white text, white text on blue billboards with 30%, 50%, 75% or 100% opacity), and the presence or absence of a drop shadow. Performance was evaluated across conditions in terms of noticeability (as a measure of visibility), processing time (as a measure of legibility), and word identification accuracy to determine significant differences. Applying a drop shadow improved legibility in plain text, while billboards lowered the upper performance threshold for font size. Although billboard conditions outperformed plain text conditions, varying billboard opacity had no significant effect on processing time, word identification, or noticeability. Overall, the findings suggest that font size, text styling, and background complexity interact to influence text visibility and legibility. ... Read more

This thesis investigates whether traffic patterns captured by a cyclist's camera coincide with moments of rider-reported workload increases, and whether a simple, scalable pipeline using a single forward-facing camera can extract useful signals for workload modelling. We developed an end-to-end system that detects and tracks nearby road users in cyclist point-of-view video, computing traffic features from 984 temporal windows across three urban rides in the city of Delft.

Using a compact set of eight traffic features and logistic regression modelling, we evaluated performance through three complementary approaches. Individual feature analysis revealed consistent directional differences between low and high workload episodes, with six of eight features maintaining the same directional relationships in multivariate analysis. Ranking performance exceeded baseline expectations, achieving within-route ROC-AUC of 0.570 (0.070 above chance) and PR-AUC of 0.450 (0.080 above baseline), meeting established criteria for small effect sizes. Cross-route generalization proved more challenging, with performance dropping to ROC-AUC of 0.517 and PR-AUC of 0.401, though consistently exceeding route-specific baselines across all test routes.

Binary classification demonstrated above-baseline performance, with within-route F1 scores of 0.478 representing a 0.108 improvement over our baseline expectations. Threshold optimization increased F1 to 0.517 by improving recall from 55.0\% to 78.3\%, though cross-route binary performance remained more limited. The traffic patterns associated with high workload episodes point toward defensive cycling scenarios characterized by increased object density, complex motion patterns, and reduced cyclist velocity in dense urban environments.

These findings establish that camera-derived traffic metrics contain detectable workload-related information while highlighting significant constraints for practical deployment. The modest magnitude of detected relationships and limited cross-route transferability indicate that while the approach demonstrates feasibility, substantial development is needed before robust real-world application. This work provides quantitative evidence for traffic-workload relationships in cycling and establishes methodological foundations for future research in automated cycling safety monitoring. ... Read more

Trustworthiness in Multilingual Large Language Mod- els (MLLMs) varies across languages, often explained by differences in pretraining resources. While this associa- tion with pretraining data is well established, we hypoth- esize that typological differences between languages also influence trustworthiness. We evaluate four similarly sized MLLMs (Aya Expanse 8B, Gemma-2 9B, Ministral 7B, and Llama-3.1 8B) on trustworthiness and analyze its correla- tions with typological features such as nominal, word or- der, verbal, negation, clause structure, phonology, token overlap, language family, and script. We find moderate to very strong correlations between specific typological fea- tures and performance, though these vary across feature- criterion pairs and are present only in English language pairs. Modeling these Z-scores confirms the importance of typology, significantly reducing error in five of eight tasks. Nominal, verbal, token overlap and word order emerge as the most consistent correlators with trustworthiness differ- ences. This exploratory work reveals an expansion of estab- lished cross-lingual typological influences and underscores the complexity of multilingual language modeling. ... Read more

This master thesis explores the possibilities of using a quadruped robot for manipulating a milking cup, using only body movments. This shows the capabilities of a quadruped for manipulating an object, without the need of higher degree of freedom manipulator arms. ... Read more

With mobile robotics being applied for more and more complex applications, their autonomy should be preserved. While a lot of research is performed into the direction of failure prediction for autonomous processes or systems, the field of mobile robots has received less attention. Proactive failure prediction for mobile robots is a useful tool to prevent unwanted downtime and undesired damages. This work attempts to fill this research gap by showing the applicability of anomaly detection methods for failure prediction in the field of mobile robots. Specifically, we employ an unsupervised Variational Autoencoder to predict failures in the operational data from the Discovery Collector, a manure cleaning robot developed by Lely Industries. We elaborately showcase the feature engineering steps which yield the best performance, provide the performance of three general datasets, and state promising next steps for root cause classification which is enabled by accurate failure prediction. All in all, our work shows that the use of feature offsets, calculated from desired values compared to actual values, enhances the model performance tremendously. The provided datasets showcase F1-scores ranging from 0.64-0.76, showing the proposed solution is able to solve the failure prediction problem in the field of mobile robots, while highlighting the encountered limitations for future improvement.
... Read more

Large Language Models (LLMs) possess significant semantic knowledge about the world, making them valuable for high level control for robots through Vision-Language-Action (VLA) models. These models integrate an LLM to deduce semantic knowledge from a robot’s vision and natural language inputs, facilitating real-world actions. Despite their potential, VLA models are a relatively new research area, with applications mostly limited to simulations or household tasks and insufficient validation in broader contexts. This study aims to develop a LLM-Enhanced Robotic Affordance and Control System (LERACS) for robotic manipulation in applied cases such as the management and maintenance of electrical grid infrastructure. LERACS is designed to visually ground manipulable objects and decompose tasks based on user instructions within a human-robot interaction chat interface, using ChatGPT. A system validation and an AI user experiment were conducted to evaluate its effectiveness in interpreting and performing actions based on pre-made and synthetically generated user instructions. These assessed LERACS’ performance across various settings and instructions. Results indicate high success rates in environmental interpretation and task execution, with robust labeling accuracy, especially in complex settings. Feedback from the AI user experiment highlighted LERACS’ adaptability, identified areas for improvement, and demonstrated its practical utility across diverse settings and task complexities. ... Read more

This study investigates the influence of synthetic data on the accuracy of 6D pose estimation in RGB images compared to RGB-Depth image-based methods. Additionally, it aims to examine how this performance varies across different types of small chess pieces during a picking task with a robotic arm. The methodology involves 3D scanning the chess pieces and generating a dataset of 61,910 synthetic images with diverse domain randomization. Using this synthetic dataset, six unique 6D pose estimation models were trained for each type of chess piece. The models were validated using a real-world 6D pose estimation dataset, and the obtained results were compared with those from the estimations on a synthetic dataset. It was observed that synthetic data can be used for bridging the visual simulation to reality gap. However there is superior performance on synthetic data compared to real-world data. This implies that results obtained in a synthetic environment cannot be directly projected to real-world scenarios. Also a noticeable decrease in accuracy in both object detection and 6D pose estimation was observed with respect to camera distance for real-world images, primarily linked to the reduced size of chess pieces in the images. Notably, each chess piece exhibited improvement after depth refinement, with accuracy closely linked to the performance of the depth camera. A picking experiment was also conducted, revealing that models relying solely on RGB data achieved a positive grasping rate of approximately 52%, while RGB-Depth-based methods reached around 70%. The findings underscore the potential for successfully picking chess pieces with and without depth refinement, emphasizing the feasibility of bridging the visual simulation to reality gap. Additionally, the study suggests several avenues for future research, including comparing synthetic data with real-world data, further exploring the training process, and introducing domain randomization in the picking experiment, such as background changes or different distractors. Furthermore, it suggests investigating diverse approaches to improve depth accuracy. ... Read more

Understanding individual differences in manual control cybernetics plays a principal role in personalizing human-machine systems. While much of the work in cybernetics utilizes models for the average controller, individualized models have also been explored, albeit constrained by data availability and the effectiveness of identification techniques. It has previously been theorized that human controller behavior may be defined by a cost function that the controller minimizes for a given control task. This cost function comprises weighted performance and effort terms, and the weightings have been connected to cybernetic models before, but not on an individualized basis. A proposed connection is through the equalization component of the cybernetic model, which has also been outlined as an indicator of individual differences. This claim was reinforced by the analysis of previously collected experiment data in this paper: Across three independent experiments, the equalization parameters, on average, showed 33.84% higher variation than the others and a greater impact on the cost function. Thus, this paper presents a human-controller-cost-function-centered approach to predict and generate individualized sets of cybernetic parameters. Individual cost function weightings were determined using the identified equalization parameters from earlier human-in-the-loop experiments. This process exhibited clear groupings in participants, as clusters of participants with similar equalization were not identified adequately. Individual cost function weightings were then employed to predict full sets of cybernetic parameters. The results indicated that the utilized cost function is incapable of reflecting the physical limitations of the human controller, as these parameters (time delay, neuromuscular natural frequency, and neuromuscular damping coefficient) showed 56.06%, -94.43%, and -170.34% deviation compared to the experimental values. Hence, a hybrid cybernetic data augmentation method was devised, yielding signal values within 15% of experimental data. ... Read more

Robots can aid in post-stroke motor function recovery and motor learning through the use of haptic feedback during collaborative training. A clear objective in robotic-assisted motor learning is to adapt the haptic feedback to individual users, but personal characteristics are not yet considered in this adaptation.

We investigated the suitability of a haptic guidance feedback strategy, based on participants’ locus of control character trait, compared to training without haptic guidance. For this purpose, a motor learning experiment was conducted on 42 healthy participants, where the internal dynamics of a pendulum had to be learned in order to hit upcoming targets. For two groups, training either with or without haptic guidance, we assessed motor learning and its generalization to similar tasks through target hitting performance, as well as behavior during training and perceived user experience.

Evidence was found of a relatively better performance improvement in both training and long-term (generalization of) motor learning for participants with a more internal compared to external locus of control. Lower observed interaction force during training and increasingly better performance throughout training
in these participants may have caused these motor learning differences. More positive user experience in these individuals through a higher perceived control over the pendulum and lower perceived frustration with haptic guidance may have also contributed.

Combined, this suggests an intrinsically better compatibility
with haptic guidance for people with a more internal rather than
external locus of control, for motor learning, during training and
in user experience ... Read more

As the global population continues to increase, so does its dependance on food, which has led to the development of more efficient food production methods. Greenhouse horticulture and genetically modified seeds have increased crop yield and decreased water and energy consumption. However, delicate greenhouse-grown vegetables continue to necessitate manual labour. The purpose of this thesis is to design and evaluate a vision system capable of detecting the spatial structure of greenhouse vegetation in order to automate plant repositioning, pruning, and leaf removal. The system employs a robotic arm and a single camera to generate custom plant features, match them, and generate a 3D model of the plant’s structure. The thesis discusses the system’s requirements, design, and evaluation, including the segmentation of plant outlines using deep learning networks, the construction of a 2D plant structure using constrained Delaunay triangulation, and triangulation-based depth calculation. Analysis of three-dimensional precision reveals a correlation between camera positional errors and the algorithm’s depth estimation. The findings support the viability of using a vision system to automate greenhouse duties if certain modifications are made. Contributing to ongoing efforts to increase greenhouse agriculture’s efficiency and reduce manual labour. ... Read more

Autonomous vehicles are becoming increasingly prevalent in society, but the transition from active driver to passive passenger is known to increase the risk and severity of motion sickness. Motion anticipation is a critical factor in this difference, and visual information is known to be a major contributor to motion anticipation. However, the underlying mechanisms of how visual information influences motion sickness remain largely unknown. This study aims to investigate the effect of visual translation on motion sickness development. To accomplish this, a proposal was made to extend an existing motion sickness prediction model with visual translational components. A human-out-of-the-loop experiment was conducted in a moving-base simulator to test the proposed model's reliability and determine whether visual translation influences motion sickness. The experiment simulated an urban ride that featured repetitive accelerating and breaking maneuvers on a straight road. Eighteen participants were subjected to three different visual conditions that varied in the amount of global optical flow, while the simulator's motion was identical. The results suggest that congruent visual translation may lead to a minor, non-significant reduction of motion sickness, while the amount of optical flow does not seem to affect it. The proposed motion sickness model was able to predict a slight decrease in motion sickness for conditions with visual optical flow, as demonstrated by the experiment. Nonetheless, it is still debatable whether congruent visual translation may influence the onset and severity of motion sickness, as the results of this study could not confirm this. ... Read more

When a person makes a decision, it is automatically accompanied by a subjective probability judgement of the decision being correct, in other words, a (local) confidence judgement. Confidence judgements have, among other things, an
effect on justifications of future decisions and behaviour. A better understanding of the metacognitive processes responsible for these confidence judgements could improve behaviour models. To date, confidence judgements are mostly studied in a fundamental manner. Little to no research has been done into confidence in more dynamic tasks. Such applied research could render insights on whether fundamental principles also hold for real-life tasks. It could also have practical relevance for several applications. Driving is amongst the areas for which an improved understanding of the decision making and accompanied confidence judgements can be useful, for instance in order to improve driving assistance systems. However, current studies on driving behaviour are merely focused on decision making and do not take confidence into account.
In this study, we made a first attempt of connecting these two fields of research by investigating the confidence of drivers in left-turn gap acceptance decisions in a driver simulator experiment (N=17). The study showed that confidence can be related to the gap size with respect to the oncoming vehicle, described by the time-to-arrival and the distance gap. Confidence increases with the gap size for gap accepting decisions and decreases with the gap size for gap rejecting decisions. In addition, we concluded that confidence can be related to the driving behaviour, and that confidence is negatively related to the decision response time. Moreover, we found that confidence judgements can best be captured with the use of an extended dynamic drift diffusion decision model of which the drift rate of the evidence accumulator as well as the decision boundaries are functions of the time-to-arrival and distance gap. Furthermore, we demonstrated that allowing for post-decision evidence accumulation in the model increases its ability to describe confidence judgements in gap rejecting decisions. Overall, the study confirmed that principles known from fundamental confidence research can be used to describe confidence judgements in a dynamic and applied task. ... Read more

This study explored several anticipatory interfaces as motion sickness mitigation tool in autonomously driving vehicles. The interfaces were designed to lower the mismatch between the expected motion and the sensory input in a moving body, by providing anticipatory information on upcoming vehicle motion. The two proposed interfaces were a Light only interface and a Combined interface featuring both light and sound. The interfaces present information on the direction and magnitude proportional to the upcoming vehicle accelerations, 0.9s before the actual event. The interfaces were tested in separate trials and compared to a Baseline trial. The comparison was done by a mixed-order, within-participants experiment (n=18), performed in the SIMONA Research Simulator, with identical 30-minute driving conditions. To test the interfaces’ effectiveness in the most motion sickness-triggering setting, subjects got a reading task on a handheld tablet. The results were quantified on an 11-point motion sickness scale and subjective survey questions. Improvements in motion sickness and subjective experience were found with both interfaces. However, only the Combined interface trial showed significant improvements compare to the Baseline. Neither of the interfaces was considered intrusive, while only the Combined interface was considered comfortable. The survey also clearly favored the Combined interface in scores on timing cor- rectness, intuitiveness, and subjective cue correctness. Although the Combined interface had the best results, strong individual preferences were reported for both interfaces, a hypothetical sound interface, or no interface. Further research is advised to focus on the Combined interface while considering optional light and/or sound disengagement in line with personal preference. ... Read more

Different paradigms can be used to evoke brainwaves. These brainwaves can be interpreted as commands that can be used to control different applications. These frameworks that interpret the brainwaves are called brain-computer interfaces. Steady-state visually evoked potentials is one of these paradigms that uses external stimuli flickering at fixed frequencies to evoke brainwaves. This paradigm is fast to issue, is reliable, and needs no training time of the user. However, to be able to distinguish multiple commands it is important to distinguish between multiple commands reliably. The fundamental metric that determines the signal quality is the signal-to-noise ratio. The signal-to-noise ratio is measured in decibels and is the ratio in power between the signal that is evoked around the stimulus frequency and the power of some baseline, referred to as noise.
To extract the power the discrete Fourier transform is calculated from the measured brainwaves. The brainwaves are measured in millivolts over time. These brainwaves can be recorded using implants in the head named intracortical or external, such as electroencephalography. External methods such as electroencephalography are much safer for the user.

Important to recognize is that one of the contributing factors to signal-to-noise ratio are the characteristics of the external stimuli displayed. Research is still trying to figure out the exact relationship between stimuli characteristics and signal-to-noise ratio. However, the experiments of previous research lack the context of the gaze of the subjects to explain the electroencephalography recordings. In this research, this is attempted to be solved using eye tracking. Furthermore, there is a research gap in the
the scientific field surrounding signal-to-noise ratio and stimuli characteristics as the effect of surrounding
stimuli on the measured signal-to-noise ratio of the target stimulus has never been investigated.

This research attempted to solve these problems by performing 2 experiments with 6 participants. One experiment shows a single stimulus across various shapes (triangles, squares, and circles), colors (red, green, and white), frequencies (9, 13, 19, and 25Hz), and sizes (10.000, 20.000, 30.000 pixels). The other experiment simulates the natural environment of the external stimuli across the same frequencies, colors, and shapes. The natural environment of a single stimulus is actually surrounding stimuli at different frequencies, as applications often require the ability to distinguish between multiple different commands. Thus, to state the main research question: "What is the relationship between the stimuli characteristics and the measured SNR?".
This question is answered by dissecting the effect that color, shape, size, frequency, and surrounding stimuli have on the signal-to-noise ratio.

... Read more

The identification of time-varying, adaptive behaviour of a human operator in basic manual control tasks is undoubtedly under development since most methodologies only account for time-invariant systems. Previous authors have proved that estimation techniques based on ARX structures can generally identify the HO model parameters. Nonetheless, ARX methods present several problems, such as the persistent bias in estimates that may increase due to coupled noise and system models. Therefore, a novel identification technique based on Box Jenkins models is proposed to achieve a more adequate match between the estimator structure and the HO model. The identification process can be conducted offline by the Ordinary Least Squares and Prediction Error Method, or online, when Recursive Least Squares and Recursive PEM are employed, respectively, in ARX and BJ models. The BJ estimator has excellent potential as an identification tool due to its bias reduction capabilities, as clearly shown in batch-fitting, although non-linear optimization processes decrease its convergence speed by 500%. An RPEM algorithm with forgetting factor 𝝀 = 0.99609 and first-order remnant BJ structure is implemented and tested under Monte Carlo simulation and experimental data. Recursive BJ algorithms could help to achieve the ideal identification method by diminishing the Neuro-Muscular parameter bias in ARX. ... Read more

This study investigated the human-robot interaction of three different augmented reality interfaces that enabled an operator to control a robot arm. The first control methods gives the operator complete manual control by moving a holographic object which the robot arm copies (Direct control). The second control method allows the operator to manually set holographic waypoints which the robot arm autonomously tries to execute (Waypoint control). And the third control method, gives the operator the capability to specify a target location which the robot autonomously moves towards (Command control). An experiment was designed to give 30 participants the capability to control a Franka Emika Research 3 robot arm with a Hololens. During the experiment, the participant had to use each control method to move the end-effector of the robot arm from a starting position to a goal position in 9 different environments. The 9 environments were divided into three manually created complexity levels (Easy, Medium, and Hard), which were based upon the performance of an informed RRT* path planner that was simulated in 67.500 unique environments. The experiment results showed that Command control outperformed the other two methods in the success rate, path length, and operation time. In terms of number of routes Command control had in general a lower number compared to Direct control, showcasing that Direct control has more flexibility in route selection. Furthermore, Direct control required more skill to get a similar performances in path length, operation time, and number of collisions as Command control. The design of Waypoint control made this method too unpredictable for the participants to use proficiently, which was underscored by the very low success rate, the long operation time, and large path length of this method. Comparing the performance of the control methods against the complexity levels did not show many significant differences, except for the success rate that decreased with a more complex environment. In summary, each control method will require unique features to completely unlock the possibilities of an augmented reality interface. The three control methods show different performances, each with their own strengths and weaknesses. And the choice of a control method is therefore dependent on the task at hand. ... Read more

Electrovibration offers potential to enrich virtual touch experiences with authentic tactile sensations on touchscreens. In controlled environments, responses to tactile stimuli may be anticipated, yet this predictability becomes uncertain in unconstrained settings due to dynamic factors like varied applied force, finger scanning speed, and sensory adaptation. To address this issue, we conducted a psychophysical study with 21 participants to investigate the effect of tactile
rendering parameters on user exploratory behaviour and perception during unconstrained exploration of artificial textures, aiming to discern a predominant tendency of interaction. Our results revealed, signal amplitude shapes human tactile
perception considerably during unconstrained exploration. We also observed, higher signal amplitudes were associated with lower finger scanning speeds, a trend tempered by significant individual differences, thereby affecting its practical effect. In contrast, the measured applied normal force and obtained finger movement pattern remained consistent and were not affected by different tactile rendering parameters. Notably, the rate of change of measured lateral force was found
to be a better metric for the perceived tactile dimensions than the lateral force magnitude. These findings enhance our understanding of perception and physics of such interactions, that could be vital for designing and delivering improved
haptic feedback on electrovibration-based tactile interfaces. ... Read more