Y.B. Eisma
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
27 records found
1
Loneliness research has mainly focused on an inside-out perspective, examining how lonely individuals process social information. Less is known about the outside-in perspective: how people judge whether someone else appears lonely and which visual cues guide that judgment. Because loneliness is not always outwardly apparent, understanding how judgments of perceived loneliness are formed may help explain when loneliness becomes socially visible, when it remains overlooked, and when it is misattributed. This study examines how elements of a visual scene influence perceived loneliness and attention toward the person being judged. Participants (N = 54) viewed 102 AI-generated photorealistic images (six everyday scenes × 17 versions) while their eye movements were recorded. In each scene, a baseline image showed the foreground person alone, with a neutral expression in a neutral setting. Manipulated versions of each baseline scene changed the person’s facial expression, the weather, the presence of objects, the background density, or the surrounding social situation. In the social scenes, further versions changed whether the foreground person remained neutral or reacted to the social situation. Each image was presented for 3 seconds, after which participants rated how lonely the foreground person appeared. Perceived loneliness was quantified as the rating shift from the matched baseline (absolute shift for magnitude comparisons); visual attention was quantified as dwell time percentage (DTP) on the foreground person’s face. Among scenes depicting a single person, the facial expression manipulation produced the largest mean absolute rating shift and the highest face DTP of the four manipulation types. Changing the weather also produced relatively large mean absolute rating shifts, though these were not consistent across the weather variations, and reduced face DTP relative to baseline. The presence of objects and cluttered backgrounds mainly reduced face DTP without corresponding changes in perceived loneliness. In scenes depicting multiple people, the foreground person’s reaction to the surrounding social situation was associated with a stronger influence on both loneliness judgments and face DTP than the type of social situation itself: reaction to the social situation produced larger mean absolute rating shifts and lower face DTP than remaining neutral. Overall, manipulations that produced the largest changes in loneliness judgments did not consistently produce the highest face DTP. This suggests that perceived loneliness was influenced by facial, environmental, and social information rather than attention on the face alone. However, how observers combined these cues was not directly examined here.
...
Loneliness research has mainly focused on an inside-out perspective, examining how lonely individuals process social information. Less is known about the outside-in perspective: how people judge whether someone else appears lonely and which visual cues guide that judgment. Because loneliness is not always outwardly apparent, understanding how judgments of perceived loneliness are formed may help explain when loneliness becomes socially visible, when it remains overlooked, and when it is misattributed. This study examines how elements of a visual scene influence perceived loneliness and attention toward the person being judged. Participants (N = 54) viewed 102 AI-generated photorealistic images (six everyday scenes × 17 versions) while their eye movements were recorded. In each scene, a baseline image showed the foreground person alone, with a neutral expression in a neutral setting. Manipulated versions of each baseline scene changed the person’s facial expression, the weather, the presence of objects, the background density, or the surrounding social situation. In the social scenes, further versions changed whether the foreground person remained neutral or reacted to the social situation. Each image was presented for 3 seconds, after which participants rated how lonely the foreground person appeared. Perceived loneliness was quantified as the rating shift from the matched baseline (absolute shift for magnitude comparisons); visual attention was quantified as dwell time percentage (DTP) on the foreground person’s face. Among scenes depicting a single person, the facial expression manipulation produced the largest mean absolute rating shift and the highest face DTP of the four manipulation types. Changing the weather also produced relatively large mean absolute rating shifts, though these were not consistent across the weather variations, and reduced face DTP relative to baseline. The presence of objects and cluttered backgrounds mainly reduced face DTP without corresponding changes in perceived loneliness. In scenes depicting multiple people, the foreground person’s reaction to the surrounding social situation was associated with a stronger influence on both loneliness judgments and face DTP than the type of social situation itself: reaction to the social situation produced larger mean absolute rating shifts and lower face DTP than remaining neutral. Overall, manipulations that produced the largest changes in loneliness judgments did not consistently produce the highest face DTP. This suggests that perceived loneliness was influenced by facial, environmental, and social information rather than attention on the face alone. However, how observers combined these cues was not directly examined here.
Evaluating LLM-Based Extraction of Meta-Analytic Data from Scientific Papers
Testing input format and prompt structure on the social robotics learning literature
This study evaluates recent large language models for extracting meta-analytic data from scientific articles in which relevant results are reported in text, tables, and figures. The study compares GPT-5.2, Claude Opus 4.6, Gemini 3.1 Pro Preview, and Gemini 3.1 Flash Lite Preview across three input representations: the original PDF, structured Markdown derived from the PDF, and a combined input consisting of the PDF together with cropped tables and figures. The final analyzed corpus contains 56 papers from a social robotics meta-analysis and includes 193 statistical data rows for which pre- and post-intervention values had to be extracted. Each row was scored on five numerical fields relevant for effect size calculation (Pre-Mean, Pre-SD, Post-Mean, Post-SD, and n), yielding 965 scored cells in total. Model outputs were compared against manually corrected ground-truth values from the original meta-analysis, which were verified against the source papers. Predicted rows were first matched to the corresponding ground-truth rows, after which the individual numerical values were scored using predefined numerical tolerances. This design allowed extraction errors to be interpreted not only as aggregate model failures, but also in relation to source format, input representation, target-row construction, and numerical reading. Across the full evaluation corpus, Markdown was often the strongest or near-strongest input representation overall, although the strongest representation differed across text-, table-, and figure-dominant papers. The highest overall performance was achieved by Claude Opus 4.6 with Markdown input. Papers in which the target values had to be extracted mainly from figures were the most difficult. A diagnostic follow-up on 17 papers that repeatedly showed errors in identifying the correct paper-condition combinations found that performance improved when the relevant condition was specified in advance and the model only had to extract the corresponding values. This suggests that many remaining errors were associated with identifying the correct extraction target row rather than with numerical reading alone.
...
This study evaluates recent large language models for extracting meta-analytic data from scientific articles in which relevant results are reported in text, tables, and figures. The study compares GPT-5.2, Claude Opus 4.6, Gemini 3.1 Pro Preview, and Gemini 3.1 Flash Lite Preview across three input representations: the original PDF, structured Markdown derived from the PDF, and a combined input consisting of the PDF together with cropped tables and figures. The final analyzed corpus contains 56 papers from a social robotics meta-analysis and includes 193 statistical data rows for which pre- and post-intervention values had to be extracted. Each row was scored on five numerical fields relevant for effect size calculation (Pre-Mean, Pre-SD, Post-Mean, Post-SD, and n), yielding 965 scored cells in total. Model outputs were compared against manually corrected ground-truth values from the original meta-analysis, which were verified against the source papers. Predicted rows were first matched to the corresponding ground-truth rows, after which the individual numerical values were scored using predefined numerical tolerances. This design allowed extraction errors to be interpreted not only as aggregate model failures, but also in relation to source format, input representation, target-row construction, and numerical reading. Across the full evaluation corpus, Markdown was often the strongest or near-strongest input representation overall, although the strongest representation differed across text-, table-, and figure-dominant papers. The highest overall performance was achieved by Claude Opus 4.6 with Markdown input. Papers in which the target values had to be extracted mainly from figures were the most difficult. A diagnostic follow-up on 17 papers that repeatedly showed errors in identifying the correct paper-condition combinations found that performance improved when the relevant condition was specified in advance and the model only had to extract the corresponding values. This suggests that many remaining errors were associated with identifying the correct extraction target row rather than with numerical reading alone.
An Adversarial Deep Learning Model Network for Mu-Rhythm Specific Neurofeedback Training
Deep Learning for EEG Neurofeedback Training
This thesis presents a novel neurofeedback system for mu-rhythm modulation using an adversarial deep learning approach. The goal was to train subjects to modulate the mu-rhythm in their brain activity and to investigate the usability of this system for reflex modulation experiments. Two EEG classifiers were implemented: a Rest vs. Motor Imagery (RestMI) model and a Motor Imagery vs. Motor Movement (MIMM) discriminator. Five healthy subjects participated in five sessions of BCI training followed by a reflex assessment. During the reflex assessment the subjects had to hold a constant flexion in their wrist in order to provoke mechanical reflexes, which introduced an extra challenge for the classifiers. The RestMI model achieved a mean classification accuracy of 0.73 in the first two sessions, however performance decreased when trials with wrist flexion were introduced. The MIMM model showed a low online performance during early sessions, indicating subjects could deceive the discriminator. The reflex assessment showed mixed results, with indication of modulation of the long latency response. These findings suggest adversarial DL models can support specific mu-rhythm training in some subjects, although further work is needed with a larger sample size and more task-specific training sessions.
...
This thesis presents a novel neurofeedback system for mu-rhythm modulation using an adversarial deep learning approach. The goal was to train subjects to modulate the mu-rhythm in their brain activity and to investigate the usability of this system for reflex modulation experiments. Two EEG classifiers were implemented: a Rest vs. Motor Imagery (RestMI) model and a Motor Imagery vs. Motor Movement (MIMM) discriminator. Five healthy subjects participated in five sessions of BCI training followed by a reflex assessment. During the reflex assessment the subjects had to hold a constant flexion in their wrist in order to provoke mechanical reflexes, which introduced an extra challenge for the classifiers. The RestMI model achieved a mean classification accuracy of 0.73 in the first two sessions, however performance decreased when trials with wrist flexion were introduced. The MIMM model showed a low online performance during early sessions, indicating subjects could deceive the discriminator. The reflex assessment showed mixed results, with indication of modulation of the long latency response. These findings suggest adversarial DL models can support specific mu-rhythm training in some subjects, although further work is needed with a larger sample size and more task-specific training sessions.
GaussianFusion++
Adaptive Radar–Camera Fusion for Maritime 3D Detection
Autonomous Surface Vehicles (ASVs) must operate safely in dynamic and often cluttered maritime environments such as ports and inland waterways. Achieving reliable situational awareness in these settings remains challenging due to the limited availability of annotated datasets, the complexity of multi-sensor alignment, and the degradation of monocular depth estimation at long range. This thesis addresses these challenges by developing a multi-modal 3D object detection framework that combines radar and camera data to improve perception robustness and depth accuracy.
To support supervised learning and evaluation, a custom dataset was constructed using real-world maritime sensor data, including radar, camera, and navigation information. Three-dimensional object annotations were manually created to capture static and moving targets across a wide range of distances.
Other core contributions include a probabilistic radar association strategy that models uncertainty in sensor measurements, a learnable Radar Gaussian Parameter Network (RGPNet) for dynamic estimation of radar association parameters, and a depth-aware mechanism for dynamically adjusting the region used for radar–camera fusion.
These methods are designed to improve object localisation performance, particularly in conditions where sensor data is sparse, noisy, or spatially misaligned.
The proposed models were evaluated through a series of controlled experiments examining detection performance across different depth intervals and clutter conditions. The results show that probabilistic radar fusion improves robustness to noise and depth estimation errors, while depth-aware association strategies enhance localisation at long range without sacrificing near-field precision. Incorporating adaptive mechanisms into the fusion process was shown to be particularly effective in scenarios involving large depth variation or limited visual information.
Overall, the findings demonstrate that uncertainty-aware, adaptive fusion methods can significantly improve 3D object detection performance for ASVs. The approaches developed in this work offer a robust foundation for future research on scalable, data-efficient maritime perception systems for autonomous navigation in complex environments. ...
To support supervised learning and evaluation, a custom dataset was constructed using real-world maritime sensor data, including radar, camera, and navigation information. Three-dimensional object annotations were manually created to capture static and moving targets across a wide range of distances.
Other core contributions include a probabilistic radar association strategy that models uncertainty in sensor measurements, a learnable Radar Gaussian Parameter Network (RGPNet) for dynamic estimation of radar association parameters, and a depth-aware mechanism for dynamically adjusting the region used for radar–camera fusion.
These methods are designed to improve object localisation performance, particularly in conditions where sensor data is sparse, noisy, or spatially misaligned.
The proposed models were evaluated through a series of controlled experiments examining detection performance across different depth intervals and clutter conditions. The results show that probabilistic radar fusion improves robustness to noise and depth estimation errors, while depth-aware association strategies enhance localisation at long range without sacrificing near-field precision. Incorporating adaptive mechanisms into the fusion process was shown to be particularly effective in scenarios involving large depth variation or limited visual information.
Overall, the findings demonstrate that uncertainty-aware, adaptive fusion methods can significantly improve 3D object detection performance for ASVs. The approaches developed in this work offer a robust foundation for future research on scalable, data-efficient maritime perception systems for autonomous navigation in complex environments. ...
Autonomous Surface Vehicles (ASVs) must operate safely in dynamic and often cluttered maritime environments such as ports and inland waterways. Achieving reliable situational awareness in these settings remains challenging due to the limited availability of annotated datasets, the complexity of multi-sensor alignment, and the degradation of monocular depth estimation at long range. This thesis addresses these challenges by developing a multi-modal 3D object detection framework that combines radar and camera data to improve perception robustness and depth accuracy.
To support supervised learning and evaluation, a custom dataset was constructed using real-world maritime sensor data, including radar, camera, and navigation information. Three-dimensional object annotations were manually created to capture static and moving targets across a wide range of distances.
Other core contributions include a probabilistic radar association strategy that models uncertainty in sensor measurements, a learnable Radar Gaussian Parameter Network (RGPNet) for dynamic estimation of radar association parameters, and a depth-aware mechanism for dynamically adjusting the region used for radar–camera fusion.
These methods are designed to improve object localisation performance, particularly in conditions where sensor data is sparse, noisy, or spatially misaligned.
The proposed models were evaluated through a series of controlled experiments examining detection performance across different depth intervals and clutter conditions. The results show that probabilistic radar fusion improves robustness to noise and depth estimation errors, while depth-aware association strategies enhance localisation at long range without sacrificing near-field precision. Incorporating adaptive mechanisms into the fusion process was shown to be particularly effective in scenarios involving large depth variation or limited visual information.
Overall, the findings demonstrate that uncertainty-aware, adaptive fusion methods can significantly improve 3D object detection performance for ASVs. The approaches developed in this work offer a robust foundation for future research on scalable, data-efficient maritime perception systems for autonomous navigation in complex environments.
To support supervised learning and evaluation, a custom dataset was constructed using real-world maritime sensor data, including radar, camera, and navigation information. Three-dimensional object annotations were manually created to capture static and moving targets across a wide range of distances.
Other core contributions include a probabilistic radar association strategy that models uncertainty in sensor measurements, a learnable Radar Gaussian Parameter Network (RGPNet) for dynamic estimation of radar association parameters, and a depth-aware mechanism for dynamically adjusting the region used for radar–camera fusion.
These methods are designed to improve object localisation performance, particularly in conditions where sensor data is sparse, noisy, or spatially misaligned.
The proposed models were evaluated through a series of controlled experiments examining detection performance across different depth intervals and clutter conditions. The results show that probabilistic radar fusion improves robustness to noise and depth estimation errors, while depth-aware association strategies enhance localisation at long range without sacrificing near-field precision. Incorporating adaptive mechanisms into the fusion process was shown to be particularly effective in scenarios involving large depth variation or limited visual information.
Overall, the findings demonstrate that uncertainty-aware, adaptive fusion methods can significantly improve 3D object detection performance for ASVs. The approaches developed in this work offer a robust foundation for future research on scalable, data-efficient maritime perception systems for autonomous navigation in complex environments.
Assessing an Augmented Reality Surgical Simulator
A Comparative Study of Productive Failure and Productive Success for Novice Motor Skill Learning
Traditional surgical training is hampered by limited practice opportunities, arising from costly resources and restricted operating room access, and by the variable feedback of apprenticeship models, which challenges standardized skill acquisition. In this study, we evaluated the efficacy of a novel Augmented Reality (AR) application for drill-guide placement and investigated the effects of two contrasting learning frameworks: Productive Success (PS), which gradually increases the difficulty as the learner’s competence grow, and Productive Failure (PF), which intentionally introduces challenges early in training to promote deeper learning. We conducted a randomized controlled trial with 16 novice participants assigned to either learning framework (eight participants per each framework), completing both training and, after at least 24 hours, retention sessions. We assessed performance using Completion Time and Success Rate, alongside subjective measures of perceived workload (NASA-TLX) and user experience (UEQ). The results demonstrated that the AR application was highly effective at reducing Completion Time; both groups significantly reduced their Completion Time from their initial baseline to the retention session, and these skills were retained. However, a corresponding significant improvement in Success Rate was not observed within the study’s timeframe. Critically, we found no significant differences between the PS and PF groups at the retention session regarding final performance levels, perceived workload, or user experience. These findings contribute valuable evidence to the field of surgical education. Firstly, they validate the use of this AR application as an effective tool for rapidly building and retaining motor skills in novices. Secondly, they suggest that for this visuomotor task in an AR environment, the theoretical benefits of PF over PS did not manifest, with both strategies leading to comparable learning outcomes. This underscores the importance of investigating instructional designs specifically tailored to the unique context of AR-based motor training. Index Terms—Augmented Reality, Surgical Simulation, Productive Failure, Motor Learning, Instructional Design.
...
Traditional surgical training is hampered by limited practice opportunities, arising from costly resources and restricted operating room access, and by the variable feedback of apprenticeship models, which challenges standardized skill acquisition. In this study, we evaluated the efficacy of a novel Augmented Reality (AR) application for drill-guide placement and investigated the effects of two contrasting learning frameworks: Productive Success (PS), which gradually increases the difficulty as the learner’s competence grow, and Productive Failure (PF), which intentionally introduces challenges early in training to promote deeper learning. We conducted a randomized controlled trial with 16 novice participants assigned to either learning framework (eight participants per each framework), completing both training and, after at least 24 hours, retention sessions. We assessed performance using Completion Time and Success Rate, alongside subjective measures of perceived workload (NASA-TLX) and user experience (UEQ). The results demonstrated that the AR application was highly effective at reducing Completion Time; both groups significantly reduced their Completion Time from their initial baseline to the retention session, and these skills were retained. However, a corresponding significant improvement in Success Rate was not observed within the study’s timeframe. Critically, we found no significant differences between the PS and PF groups at the retention session regarding final performance levels, perceived workload, or user experience. These findings contribute valuable evidence to the field of surgical education. Firstly, they validate the use of this AR application as an effective tool for rapidly building and retaining motor skills in novices. Secondly, they suggest that for this visuomotor task in an AR environment, the theoretical benefits of PF over PS did not manifest, with both strategies leading to comparable learning outcomes. This underscores the importance of investigating instructional designs specifically tailored to the unique context of AR-based motor training. Index Terms—Augmented Reality, Surgical Simulation, Productive Failure, Motor Learning, Instructional Design.
Numerous techniques have been developed in order to explain the reasoning process of black-box models. Among them is a class of models that are designed to be inherently interpretable: select-then-predict models (a.k.a. rationale-based models). These models are meant to explain their prediction by highlighting part of the input as evidence. The evidence, called the rationale, should consist of the most salient parts of the input text that contribute the most to the model's decision.
However, according to some recent studies, these models are not truly interpretable, because they do not provide faithful explanations (i.e., explanations that accurately reflect the true reasoning process of the model).
In this thesis we give a formal definition of the degree of unfaithfulness to quantify unfaithful behavior. Then, we introduce an experiment to test the faithfulness of select-then-predict models and prove that select-then-predict models can provide unfaithful rationales. Lastly, we introduce a loss function, which we call the unfaithfulness loss, which minimizes the degree of unfaithfulness of select-then-predict models and teaches them to produce more faithful and plausible rationales. ...
However, according to some recent studies, these models are not truly interpretable, because they do not provide faithful explanations (i.e., explanations that accurately reflect the true reasoning process of the model).
In this thesis we give a formal definition of the degree of unfaithfulness to quantify unfaithful behavior. Then, we introduce an experiment to test the faithfulness of select-then-predict models and prove that select-then-predict models can provide unfaithful rationales. Lastly, we introduce a loss function, which we call the unfaithfulness loss, which minimizes the degree of unfaithfulness of select-then-predict models and teaches them to produce more faithful and plausible rationales. ...
Numerous techniques have been developed in order to explain the reasoning process of black-box models. Among them is a class of models that are designed to be inherently interpretable: select-then-predict models (a.k.a. rationale-based models). These models are meant to explain their prediction by highlighting part of the input as evidence. The evidence, called the rationale, should consist of the most salient parts of the input text that contribute the most to the model's decision.
However, according to some recent studies, these models are not truly interpretable, because they do not provide faithful explanations (i.e., explanations that accurately reflect the true reasoning process of the model).
In this thesis we give a formal definition of the degree of unfaithfulness to quantify unfaithful behavior. Then, we introduce an experiment to test the faithfulness of select-then-predict models and prove that select-then-predict models can provide unfaithful rationales. Lastly, we introduce a loss function, which we call the unfaithfulness loss, which minimizes the degree of unfaithfulness of select-then-predict models and teaches them to produce more faithful and plausible rationales.
However, according to some recent studies, these models are not truly interpretable, because they do not provide faithful explanations (i.e., explanations that accurately reflect the true reasoning process of the model).
In this thesis we give a formal definition of the degree of unfaithfulness to quantify unfaithful behavior. Then, we introduce an experiment to test the faithfulness of select-then-predict models and prove that select-then-predict models can provide unfaithful rationales. Lastly, we introduce a loss function, which we call the unfaithfulness loss, which minimizes the degree of unfaithfulness of select-then-predict models and teaches them to produce more faithful and plausible rationales.
End-to-end behavior cloning agent for an object handover task
Training and evaluating a robot to perform simulated human-to-robot object handovers without requiring hand-object segmentation
Current visuomotor manipulators jointly train their perception-planning-action components simultaneously using an end-to-end framework to avoid hand-engineering components. Despite this, methods for human-to-robot object handover tasks require a perception component that segments the hand from the object, which can introduce error propagation. For this reason, this study investigates the applicability of an end-to-end framework that eliminates the need for hand-object segmentation in a simulated human-to-robot object handover task using HandoverSim.
To address this, a behavior cloning agent is used to convert camera input into RGB-D voxel space and output discretized 6-DoF manipulation to directly discover features for the handover task. This study introduces a framework that combines the behavior cloning agent with the HandoverSim, which allows experimenting with various training configurations. These configurations consist of experiments with: 1) expert demonstration data; 2) optimal camera setup; 3) handover objects; and 4) voxel-based RGB augmentation techniques.
The trained model is evaluated on its generalization to diverse handover conditions in the HandoverSim Benchmark. The results demonstrate that the behavior cloning agent can learn features for the handover task without requiring a perception component. The model learns the grasp-object relation whilst minimizing contact with the hand. Despite this, performance is limited by sparse training data and grasping accuracy. ...
To address this, a behavior cloning agent is used to convert camera input into RGB-D voxel space and output discretized 6-DoF manipulation to directly discover features for the handover task. This study introduces a framework that combines the behavior cloning agent with the HandoverSim, which allows experimenting with various training configurations. These configurations consist of experiments with: 1) expert demonstration data; 2) optimal camera setup; 3) handover objects; and 4) voxel-based RGB augmentation techniques.
The trained model is evaluated on its generalization to diverse handover conditions in the HandoverSim Benchmark. The results demonstrate that the behavior cloning agent can learn features for the handover task without requiring a perception component. The model learns the grasp-object relation whilst minimizing contact with the hand. Despite this, performance is limited by sparse training data and grasping accuracy. ...
Current visuomotor manipulators jointly train their perception-planning-action components simultaneously using an end-to-end framework to avoid hand-engineering components. Despite this, methods for human-to-robot object handover tasks require a perception component that segments the hand from the object, which can introduce error propagation. For this reason, this study investigates the applicability of an end-to-end framework that eliminates the need for hand-object segmentation in a simulated human-to-robot object handover task using HandoverSim.
To address this, a behavior cloning agent is used to convert camera input into RGB-D voxel space and output discretized 6-DoF manipulation to directly discover features for the handover task. This study introduces a framework that combines the behavior cloning agent with the HandoverSim, which allows experimenting with various training configurations. These configurations consist of experiments with: 1) expert demonstration data; 2) optimal camera setup; 3) handover objects; and 4) voxel-based RGB augmentation techniques.
The trained model is evaluated on its generalization to diverse handover conditions in the HandoverSim Benchmark. The results demonstrate that the behavior cloning agent can learn features for the handover task without requiring a perception component. The model learns the grasp-object relation whilst minimizing contact with the hand. Despite this, performance is limited by sparse training data and grasping accuracy.
To address this, a behavior cloning agent is used to convert camera input into RGB-D voxel space and output discretized 6-DoF manipulation to directly discover features for the handover task. This study introduces a framework that combines the behavior cloning agent with the HandoverSim, which allows experimenting with various training configurations. These configurations consist of experiments with: 1) expert demonstration data; 2) optimal camera setup; 3) handover objects; and 4) voxel-based RGB augmentation techniques.
The trained model is evaluated on its generalization to diverse handover conditions in the HandoverSim Benchmark. The results demonstrate that the behavior cloning agent can learn features for the handover task without requiring a perception component. The model learns the grasp-object relation whilst minimizing contact with the hand. Despite this, performance is limited by sparse training data and grasping accuracy.
Evaluation of Driver Situational Awareness in ADAS Speed Limit Recognition
Insights from Eye Tracking and Behavioural Analysis
Advanced Driver Assistance Systems (ADAS) play a crucial role in enhancing road safety and driver comfort by providing real-time alerts and support. However, the effectiveness of ADAS signals in influencing driver situational awareness (SA) remains an area requiring further investigation. This study examines how different ADAS speed limit alert designs impact the three levels of SA: perception, comprehension, and projection.
A real-world driving experiment was conducted in Amersfoort along a 4.3 km route, comparing two ADAS systems with distinct signal designs. System A provided both the current and upcoming speed limits, while System B displayed only the current limit. Data collection included eye-tracking for SA1 (perception), self-reported questionnaires for SA2 (comprehension), and speed analysis for SA3 (projection). Statistical analyses, including significance tests, correlation analysis, and Linear Mixed Models (LMM), were employed to evaluate the influence of ADAS design and driver-related factors on SA.
Results indicate that while most SA indicators did not show significant differences between the two ADAS systems, System A consistently demonstrated advantages in visual attention allocation and comprehension. Additionally, driver experience and familiarity with ADAS were found to influence SA, with experienced drivers exhibiting reduced cognitive load and faster responses. A weak correlation between different SA levels suggests that drivers integrate ADAS alerts with external environmental cues.
This study highlights the importance of designing ADAS alerts that provide anticipatory information to enhance driver awareness and decision-making. Future research should expand sample sizes, incorporate additional SA indicators, and explore the long-term effects of ADAS exposure. These findings offer valuable insights for optimizing ADAS signal design to improve road safety and driving performance. ...
A real-world driving experiment was conducted in Amersfoort along a 4.3 km route, comparing two ADAS systems with distinct signal designs. System A provided both the current and upcoming speed limits, while System B displayed only the current limit. Data collection included eye-tracking for SA1 (perception), self-reported questionnaires for SA2 (comprehension), and speed analysis for SA3 (projection). Statistical analyses, including significance tests, correlation analysis, and Linear Mixed Models (LMM), were employed to evaluate the influence of ADAS design and driver-related factors on SA.
Results indicate that while most SA indicators did not show significant differences between the two ADAS systems, System A consistently demonstrated advantages in visual attention allocation and comprehension. Additionally, driver experience and familiarity with ADAS were found to influence SA, with experienced drivers exhibiting reduced cognitive load and faster responses. A weak correlation between different SA levels suggests that drivers integrate ADAS alerts with external environmental cues.
This study highlights the importance of designing ADAS alerts that provide anticipatory information to enhance driver awareness and decision-making. Future research should expand sample sizes, incorporate additional SA indicators, and explore the long-term effects of ADAS exposure. These findings offer valuable insights for optimizing ADAS signal design to improve road safety and driving performance. ...
Advanced Driver Assistance Systems (ADAS) play a crucial role in enhancing road safety and driver comfort by providing real-time alerts and support. However, the effectiveness of ADAS signals in influencing driver situational awareness (SA) remains an area requiring further investigation. This study examines how different ADAS speed limit alert designs impact the three levels of SA: perception, comprehension, and projection.
A real-world driving experiment was conducted in Amersfoort along a 4.3 km route, comparing two ADAS systems with distinct signal designs. System A provided both the current and upcoming speed limits, while System B displayed only the current limit. Data collection included eye-tracking for SA1 (perception), self-reported questionnaires for SA2 (comprehension), and speed analysis for SA3 (projection). Statistical analyses, including significance tests, correlation analysis, and Linear Mixed Models (LMM), were employed to evaluate the influence of ADAS design and driver-related factors on SA.
Results indicate that while most SA indicators did not show significant differences between the two ADAS systems, System A consistently demonstrated advantages in visual attention allocation and comprehension. Additionally, driver experience and familiarity with ADAS were found to influence SA, with experienced drivers exhibiting reduced cognitive load and faster responses. A weak correlation between different SA levels suggests that drivers integrate ADAS alerts with external environmental cues.
This study highlights the importance of designing ADAS alerts that provide anticipatory information to enhance driver awareness and decision-making. Future research should expand sample sizes, incorporate additional SA indicators, and explore the long-term effects of ADAS exposure. These findings offer valuable insights for optimizing ADAS signal design to improve road safety and driving performance.
A real-world driving experiment was conducted in Amersfoort along a 4.3 km route, comparing two ADAS systems with distinct signal designs. System A provided both the current and upcoming speed limits, while System B displayed only the current limit. Data collection included eye-tracking for SA1 (perception), self-reported questionnaires for SA2 (comprehension), and speed analysis for SA3 (projection). Statistical analyses, including significance tests, correlation analysis, and Linear Mixed Models (LMM), were employed to evaluate the influence of ADAS design and driver-related factors on SA.
Results indicate that while most SA indicators did not show significant differences between the two ADAS systems, System A consistently demonstrated advantages in visual attention allocation and comprehension. Additionally, driver experience and familiarity with ADAS were found to influence SA, with experienced drivers exhibiting reduced cognitive load and faster responses. A weak correlation between different SA levels suggests that drivers integrate ADAS alerts with external environmental cues.
This study highlights the importance of designing ADAS alerts that provide anticipatory information to enhance driver awareness and decision-making. Future research should expand sample sizes, incorporate additional SA indicators, and explore the long-term effects of ADAS exposure. These findings offer valuable insights for optimizing ADAS signal design to improve road safety and driving performance.
Master thesis
(2024)
-
C. Qi, L. Marchal Crespo, M.L. van de Ruit, A.L. Ratschat, W. Mugge, Y.B. Eisma
Robotic rehabilitation systems that provide proprioceptive information offer a promising approach to helping stroke survivors regain lost proprioceptive functions. One potential way to provide proprioceptive information is viscosity rendering. However, how the modulated viscosity rendering affects brain activity remains unclear. To investigate the correlation between viscosity rendering and brain activity and to provide a neurological basis for the design of robotic rehabilitation systems, an experimental setup was developed to deliver various viscosity rendering and fixed stiffness rendering during hand movement. In the experiment, twelve healthy participants interacted with virtual bottles with the same bottle stiffness but filled with liquids of different viscosities under the same movement speed, providing different levels of proprioceptive information in the form of force on muscles and joints. Control conditions without viscosity and stiffness rendering were also tested, involving both passive and active hand movements at the same speed. Results showed that stronger mu-ERD and beta-ERD were observed during movements with viscosity and stiffness rendering compared to control conditions. No significant evidence suggested that different viscosity in rendering caused variations in mu-ERD or beta-ERD. Additionally, no significant differences were found between active movement without haptic rendering and passive movement in EEG activities. These findings suggest that while the existence of viscosity and stiffness rendering during movement strengthens brain activity, modulating viscosity rendering does not significantly affect this response. This insight is particularly valuable for designing robotic rehabilitation systems that incorporate viscosity rendering.
...
Robotic rehabilitation systems that provide proprioceptive information offer a promising approach to helping stroke survivors regain lost proprioceptive functions. One potential way to provide proprioceptive information is viscosity rendering. However, how the modulated viscosity rendering affects brain activity remains unclear. To investigate the correlation between viscosity rendering and brain activity and to provide a neurological basis for the design of robotic rehabilitation systems, an experimental setup was developed to deliver various viscosity rendering and fixed stiffness rendering during hand movement. In the experiment, twelve healthy participants interacted with virtual bottles with the same bottle stiffness but filled with liquids of different viscosities under the same movement speed, providing different levels of proprioceptive information in the form of force on muscles and joints. Control conditions without viscosity and stiffness rendering were also tested, involving both passive and active hand movements at the same speed. Results showed that stronger mu-ERD and beta-ERD were observed during movements with viscosity and stiffness rendering compared to control conditions. No significant evidence suggested that different viscosity in rendering caused variations in mu-ERD or beta-ERD. Additionally, no significant differences were found between active movement without haptic rendering and passive movement in EEG activities. These findings suggest that while the existence of viscosity and stiffness rendering during movement strengthens brain activity, modulating viscosity rendering does not significantly affect this response. This insight is particularly valuable for designing robotic rehabilitation systems that incorporate viscosity rendering.
Relocated haptic feedback from the fingertip to the proximal phalanx can alter the perception of physical interactions by simultaneously displaying relocated multimodal tactile cues. However, how these relocated tactile cues alter the vibrotactile sensitivity of the fingertip remains unclear. This two-site stimulation study employs a customdesigned multimodal haptic ring for the proximal phalanx to evaluate the effect of relocated cold, hot, and pressure stimuli on the vibrotactile sensitivity of the fingertip. Our results show no significant difference between the vibratory detection thresholds of six multimodal tactile conditions. In contrast to single-site multimodal stimulation, where vibrotactile sensitivity is significantly altered by pressure and thermal stimuli, these results imply the feasibility of applying thermal and pressure stimuli without significantly altering fingertip sensitivity. Relocation of tactile stimuli keeps the vibrotactile sensitivity of the fingertip intact and opens up the possibility of altering tactile perception while interacting with the physical environment, making this technology feasible for mixed reality applications.
...
Relocated haptic feedback from the fingertip to the proximal phalanx can alter the perception of physical interactions by simultaneously displaying relocated multimodal tactile cues. However, how these relocated tactile cues alter the vibrotactile sensitivity of the fingertip remains unclear. This two-site stimulation study employs a customdesigned multimodal haptic ring for the proximal phalanx to evaluate the effect of relocated cold, hot, and pressure stimuli on the vibrotactile sensitivity of the fingertip. Our results show no significant difference between the vibratory detection thresholds of six multimodal tactile conditions. In contrast to single-site multimodal stimulation, where vibrotactile sensitivity is significantly altered by pressure and thermal stimuli, these results imply the feasibility of applying thermal and pressure stimuli without significantly altering fingertip sensitivity. Relocation of tactile stimuli keeps the vibrotactile sensitivity of the fingertip intact and opens up the possibility of altering tactile perception while interacting with the physical environment, making this technology feasible for mixed reality applications.
Lely is developing a tethered manure applicator robot, designed to work on the fields of farms. This vehicle called the Jojo, is constrained in its movement due to the attached tether. It can not make tight turns and can only reverse by backtracking the driven path. This thesis explores the path planning issue and provides a novel solution. The problem is best described as the shortest path from a starting point to a set of strokes that cover the field, with a constrained turning radius. We first explore existing solutions and discover that none exist that are suitable. Further research finds that the behaviour of this vehicle is best compared with steerable needles, but at a different scale. First, we construct a simulation environment for the vehicle. This simulation is used to develop the plan execution engine, path follower and safety checks. This is necessary for testing the solutions in the simulation and in the real world. Secondly, with some inspiration from the solutions provided for the steerable needles, we arrive at a method for finding a Jojo path. The method makes use of a large graph that represents translations associated with a given location. We also provide methods for attaching the starting point and destinations in the form of strokes, either via the start or end of the strokes or via the middle. A Jojo plan is then obtained by performing a directed Steiner tree approximation on this graph, where the starting point is the root and the terminals are the strokes. In order to make the approach fast enough, we spend a substantial amount of time profiling and analysing the program and design a number of performance improvements. These allow for the approach to solve instances that are larger than necessary. Thirdly, we test, compare and improve the approach. We look at both small instances to perform numerical analysis in order to optimise a set of parameters. We then inspect larger instances that are based on real world scenarios. Lastly, we perform a set of real world tests using the vehicle. One of these tests highlights a shortcoming of the algorithm, which we discuss and provide a solution for
...
Lely is developing a tethered manure applicator robot, designed to work on the fields of farms. This vehicle called the Jojo, is constrained in its movement due to the attached tether. It can not make tight turns and can only reverse by backtracking the driven path. This thesis explores the path planning issue and provides a novel solution. The problem is best described as the shortest path from a starting point to a set of strokes that cover the field, with a constrained turning radius. We first explore existing solutions and discover that none exist that are suitable. Further research finds that the behaviour of this vehicle is best compared with steerable needles, but at a different scale. First, we construct a simulation environment for the vehicle. This simulation is used to develop the plan execution engine, path follower and safety checks. This is necessary for testing the solutions in the simulation and in the real world. Secondly, with some inspiration from the solutions provided for the steerable needles, we arrive at a method for finding a Jojo path. The method makes use of a large graph that represents translations associated with a given location. We also provide methods for attaching the starting point and destinations in the form of strokes, either via the start or end of the strokes or via the middle. A Jojo plan is then obtained by performing a directed Steiner tree approximation on this graph, where the starting point is the root and the terminals are the strokes. In order to make the approach fast enough, we spend a substantial amount of time profiling and analysing the program and design a number of performance improvements. These allow for the approach to solve instances that are larger than necessary. Thirdly, we test, compare and improve the approach. We look at both small instances to perform numerical analysis in order to optimise a set of parameters. We then inspect larger instances that are based on real world scenarios. Lastly, we perform a set of real world tests using the vehicle. One of these tests highlights a shortcoming of the algorithm, which we discuss and provide a solution for
Classification of Covert Vowels in Spanish and Dutch
What do brain signals say about inner speech?
Patients with neuromuscular diseases that are unable to speak, but whose cognitive ability has been maintained, can be benefited from Brain Computer Interfaces (BCIs). The decoding of inner (covert) speech from EEGs consists of one of the state of the art methods that aim to tackle this issue. High variability between subjects, as well as low signal to noise ratio (SNR) undermine the methods used, and introduce the need for computer assisted solutions. Thus, machine learning models as well as large amounts of recorded data are required to design effective algorithms and produce substantial results. In this study, covert vowel classification was performed in a systematic way, by making use of two openly shared databases from literature; the Coretto database, that contains EEG recordings of native Spanish speakers, and the DAIS dataset, which includes EEG recordings of native Dutch speakers. Six classifiers were initially selected to perform 5-class classification: a Random Forest (RF), a k Nearest Neighbours (kNN), a Gaussian Naive Bayers (GNB), a Deep Convolutional Neural Network (DCNN), a Shallow Convolutional Neural Network (SCNN) and a Long Short Term Memory Recurrent Neural Network (LSTM). The DCNN outperformed the other methods, with average intra-subject accuracies of 35% for Coretto and 39% for DAIS (chance level 20%). Afterwards, an Overt versus Covert trials experiment was implemented, to test the limits of overt speech decoding from EEGs. The overt result was slightly higher than covert, with an intra-subject average value of 37.8% for Coretto and 40.5% for DAIS (chance level 20%). Finally, binary classification was performed to identify those pairs of vowels that can be classified more efficiently. Vowels /a/ and /u/ seemed to perform better in average in both datasets (average of 64.8% for Coretto and 64.4% for DAIS with a chance accuracy of 50%). Future work should focus on identifying the useful parts of the EEG recordings, increasing the SNR and the resolution of the electrodes, and defining the most appropriate dictionaries of words/vowels for a BCI. Also, more studies should follow systematic ways of comparisons between datasets, to obtain less ambiguous insights and lead this field to improvements.
...
Patients with neuromuscular diseases that are unable to speak, but whose cognitive ability has been maintained, can be benefited from Brain Computer Interfaces (BCIs). The decoding of inner (covert) speech from EEGs consists of one of the state of the art methods that aim to tackle this issue. High variability between subjects, as well as low signal to noise ratio (SNR) undermine the methods used, and introduce the need for computer assisted solutions. Thus, machine learning models as well as large amounts of recorded data are required to design effective algorithms and produce substantial results. In this study, covert vowel classification was performed in a systematic way, by making use of two openly shared databases from literature; the Coretto database, that contains EEG recordings of native Spanish speakers, and the DAIS dataset, which includes EEG recordings of native Dutch speakers. Six classifiers were initially selected to perform 5-class classification: a Random Forest (RF), a k Nearest Neighbours (kNN), a Gaussian Naive Bayers (GNB), a Deep Convolutional Neural Network (DCNN), a Shallow Convolutional Neural Network (SCNN) and a Long Short Term Memory Recurrent Neural Network (LSTM). The DCNN outperformed the other methods, with average intra-subject accuracies of 35% for Coretto and 39% for DAIS (chance level 20%). Afterwards, an Overt versus Covert trials experiment was implemented, to test the limits of overt speech decoding from EEGs. The overt result was slightly higher than covert, with an intra-subject average value of 37.8% for Coretto and 40.5% for DAIS (chance level 20%). Finally, binary classification was performed to identify those pairs of vowels that can be classified more efficiently. Vowels /a/ and /u/ seemed to perform better in average in both datasets (average of 64.8% for Coretto and 64.4% for DAIS with a chance accuracy of 50%). Future work should focus on identifying the useful parts of the EEG recordings, increasing the SNR and the resolution of the electrodes, and defining the most appropriate dictionaries of words/vowels for a BCI. Also, more studies should follow systematic ways of comparisons between datasets, to obtain less ambiguous insights and lead this field to improvements.
The agricultural industry is undergoing a transformative shift towards efficiency and large-scale production. Manual labor is unable to meet the increasing demand, leading to the development of automation techniques. During production, objects are adjusted and moved along a line to maintain productivity. This work investigates the feasibility of developing a system that detects and determines the position and rotation of a small clipping object in a cluttered environment. To achieve this feasibility, a neural network has been trained on a custom dataset with images containing 6-D pose annotations of a clip that do not require manual annotations. The designed system also executes a validation step with a second set of sensors in the form of a stereo camera. The entire methodology has been evaluated on a test setup. The small and round properties of the clip cause a rotational error that does not fall in positional and rotational requirements set by a mechanical grabber on a robotic arm. Higher positional precision of annotation data and depth information from the stereo camera is necessary to make this methodology feasible.
...
The agricultural industry is undergoing a transformative shift towards efficiency and large-scale production. Manual labor is unable to meet the increasing demand, leading to the development of automation techniques. During production, objects are adjusted and moved along a line to maintain productivity. This work investigates the feasibility of developing a system that detects and determines the position and rotation of a small clipping object in a cluttered environment. To achieve this feasibility, a neural network has been trained on a custom dataset with images containing 6-D pose annotations of a clip that do not require manual annotations. The designed system also executes a validation step with a second set of sensors in the form of a stereo camera. The entire methodology has been evaluated on a test setup. The small and round properties of the clip cause a rotational error that does not fall in positional and rotational requirements set by a mechanical grabber on a robotic arm. Higher positional precision of annotation data and depth information from the stereo camera is necessary to make this methodology feasible.
Simulating Human Motor Learning
An Old Solution in New Environments
Feedback error learning (FEL) is a classical computational model that describes human motor learning. It consists of forward and inverse models representing internal dynamics and environmental disturbances. Such models can be used as controllers that represent the function of the motor cortex. On top of FEL, a model has been built with jointly trained feedforward and feedback controllers using a neural network. The controllers that actuated six muscles driving a two-degrees-of-freedom arm model were trained offline. This model successfully simulated human learning of point-to-point reaching movements in a horizontal plane when it was tasked to adapt itself in a null field (NF) and a velocity-dependent force field (VF). In this study, we further tested this model in the divergent force fields (DF) and a channelled force field (CF) to observe its performance. The comparison between the simulation results and the experimental evidence suggests that this model can predict some of the key features of the learning process, such as the kinematics, the muscle dynamics, and the impedance profiles. The learning decay was hindered when the lateral error was artificially eliminated by the CF, as reported in the literature. Overall, the model could converge towards realistic
...
Feedback error learning (FEL) is a classical computational model that describes human motor learning. It consists of forward and inverse models representing internal dynamics and environmental disturbances. Such models can be used as controllers that represent the function of the motor cortex. On top of FEL, a model has been built with jointly trained feedforward and feedback controllers using a neural network. The controllers that actuated six muscles driving a two-degrees-of-freedom arm model were trained offline. This model successfully simulated human learning of point-to-point reaching movements in a horizontal plane when it was tasked to adapt itself in a null field (NF) and a velocity-dependent force field (VF). In this study, we further tested this model in the divergent force fields (DF) and a channelled force field (CF) to observe its performance. The comparison between the simulation results and the experimental evidence suggests that this model can predict some of the key features of the learning process, such as the kinematics, the muscle dynamics, and the impedance profiles. The learning decay was hindered when the lateral error was artificially eliminated by the CF, as reported in the literature. Overall, the model could converge towards realistic
An aging population puts a pressure on health-care workers working with dementia patients globally. A potential solution is to provide care with Socially Assistive Robots (SARs), i.e. robots who help people through social interaction. However, for effective care these SARs must be able to personalize their behavior to individual patients and adapt this behavior to changes in this patient’s references. This paper presents the decision making process of a SAR that enables the SAR to personalize its behavior to the personality of the patients and adapt this behavior to their current state-of-mind. The system consists of a Fuzzy Logic Control personalization module, which personalizes the SAR’s behavior and a Reinforcement Learning based decision making module together with a Fuzzy Logic Control reward module for the adaption of the personalized behavior. The personalization of the SAR’s behavior is assessed by comparing the output of the system with answers from a survey. The average scatter index over all different behavioral parameters of the SAR is 20.4% The adaptation of the behavior is assessed with computer-based simulations, where an overall accuracy of 81.8% is achieved. A third experiment is carried out to assess the effect of adding the Fuzzy Logic Control personalization module to the system. This experiment shows that adding the personalization module to the decision making system of the SAR decreases the time for the learning process to converge with 13.3%. Although the first assessments of the system look promising, more extensive experiments should be held in later stages of the research. A crucial experiment that must be held in future research is performing real-life interactions between dementia patients and the SAR, in such an experiment the functionality of the system really can be assessed.
...
An aging population puts a pressure on health-care workers working with dementia patients globally. A potential solution is to provide care with Socially Assistive Robots (SARs), i.e. robots who help people through social interaction. However, for effective care these SARs must be able to personalize their behavior to individual patients and adapt this behavior to changes in this patient’s references. This paper presents the decision making process of a SAR that enables the SAR to personalize its behavior to the personality of the patients and adapt this behavior to their current state-of-mind. The system consists of a Fuzzy Logic Control personalization module, which personalizes the SAR’s behavior and a Reinforcement Learning based decision making module together with a Fuzzy Logic Control reward module for the adaption of the personalized behavior. The personalization of the SAR’s behavior is assessed by comparing the output of the system with answers from a survey. The average scatter index over all different behavioral parameters of the SAR is 20.4% The adaptation of the behavior is assessed with computer-based simulations, where an overall accuracy of 81.8% is achieved. A third experiment is carried out to assess the effect of adding the Fuzzy Logic Control personalization module to the system. This experiment shows that adding the personalization module to the decision making system of the SAR decreases the time for the learning process to converge with 13.3%. Although the first assessments of the system look promising, more extensive experiments should be held in later stages of the research. A crucial experiment that must be held in future research is performing real-life interactions between dementia patients and the SAR, in such an experiment the functionality of the system really can be assessed.
Master thesis
(2023)
-
J.E. van Dorth tot Medler, J.C.F. de Winter, K. Elsendoorn, Y.B. Eisma, A.E. Zaidman
Background: For rigorous software testing, integration and end-to-end tests are essential to ensure the expected behavior of multiple interacting components of the system. When software is subjected to integration or end-to-end tests, it is often unfeasible to test every code change individually, as the runtime of these tests is usually significantly larger compared to unit tests. For this reason, batches of code changes from multiple authors are often tested simultaneously. Problem: An issue with testing multiple changes simultaneously is that it can be unclear which change form which author caused the failure when tests fail, as all changes from all authors included in the test can be at fault. Design: To solve this, a new automatic fault localization algorithm called GitFL is introduced, which combines state-of-the-art fault localization with version control history information for enhanced performance. GitFL was evaluated on a C++ repository at Adyen where tests are considered to be end-to-end. Findings: It showed that the addition of version control history information significantly increases the performance of fault localization for systems where multiple changes are tested simultaneously. Societal implications: This work provides insights on improved fault localization for these systems, which could enable organizations which develop these systems to speed up their testing and development processes. Originality: This work contributes by focusing on fault localization specifically for systems where multiple changes are tested simultaneously, which was not researched before.
...
...
Background: For rigorous software testing, integration and end-to-end tests are essential to ensure the expected behavior of multiple interacting components of the system. When software is subjected to integration or end-to-end tests, it is often unfeasible to test every code change individually, as the runtime of these tests is usually significantly larger compared to unit tests. For this reason, batches of code changes from multiple authors are often tested simultaneously. Problem: An issue with testing multiple changes simultaneously is that it can be unclear which change form which author caused the failure when tests fail, as all changes from all authors included in the test can be at fault. Design: To solve this, a new automatic fault localization algorithm called GitFL is introduced, which combines state-of-the-art fault localization with version control history information for enhanced performance. GitFL was evaluated on a C++ repository at Adyen where tests are considered to be end-to-end. Findings: It showed that the addition of version control history information significantly increases the performance of fault localization for systems where multiple changes are tested simultaneously. Societal implications: This work provides insights on improved fault localization for these systems, which could enable organizations which develop these systems to speed up their testing and development processes. Originality: This work contributes by focusing on fault localization specifically for systems where multiple changes are tested simultaneously, which was not researched before.
The problem of assisting users in comprehending the robotic scenario information in a retail setting has been studied. To design the system, an integrated ontology composed of several IEEE standard ontologies and a labelled property graph (LPG)-based ontology modified from the Web Ontology Language (OWL)-based ontology was proposed to symbolize information in the robotic environment. Then, a knowledge graph (KG)-based chatbot was developed to provide natural language interaction with users. A case study in a retail setting was designed, and the results were analyzed. The effectiveness of our designed system has been experimentally validated in both static and dynamic scenarios, with at least 1.5 times improvements.
...
The problem of assisting users in comprehending the robotic scenario information in a retail setting has been studied. To design the system, an integrated ontology composed of several IEEE standard ontologies and a labelled property graph (LPG)-based ontology modified from the Web Ontology Language (OWL)-based ontology was proposed to symbolize information in the robotic environment. Then, a knowledge graph (KG)-based chatbot was developed to provide natural language interaction with users. A case study in a retail setting was designed, and the results were analyzed. The effectiveness of our designed system has been experimentally validated in both static and dynamic scenarios, with at least 1.5 times improvements.
Background: A brain-computer interface (BCI) is a system that enables humans to control a computer by their brain signals. This can be achieved by modulating the sensorimotor rhythm (SMR) through both motor execution and motor imagery. The potential enhancement of spinal reflexes and motor control through SMR training is attributed to the hypothesis that activity-dependent brain plasticity guides spinal plasticity during motor skill learning. However, the signal processing needed for conversion from raw brain signals to a robust control signal is challenging. The recorded electroencephalogram (EEG) signals are contaminated by multiple unknown sources and suffer from inter-subject variability, complicating the development of the BCI.
Objective: To obtain a robust control signal the study 1) investigated the relation between event-related desynchronization (ERD) and mechanical stretch reflex size in the flexor carpi radialis across four muscle pre-loads consisting of 0%, 5%, 25% and 40% of maximum voluntary contraction (MVC), 2) investigated the ability of three offline signal processing paradigms in distinguishing between periods of rest and activity using EEG data associated with motor execution and motor imagery, 3) built a pseudo-online signal processing paradigm to simulate real-time signal processing based on a single trial and a continuous data stream.
Method: Mechanical stretch perturbations were applied to the wrist under four percentages of MVC during motor execution and imagery conditions in six healthy subjects. The data anal- ysis encompassed signal processing techniques including pre- processing with a large Laplacian filter, feature extraction through autoregressive modelling (AR), power spectral density (PSD), or discrete wavelet transform (DWT), and classification using linear discriminant analysis (LDA).
Results: Mechanical stretch reflex sizes and ERD amplitude significantly increased with increasing percentage of MVC for motor execution trials. For motor imagery trials, no significant correlation was found between the stretch reflex size and ERD amplitude. The offline signal processing paradigms resulted in classification accuracies of 73.55% (PSD), 71.96% (DWT) and 57.13% (AR). The classification accuracies significantly increased with increasing percentage of MVC. The pseudo-online paradigm resulted in a mean classification accuracy of 51.38%.
Conclusions: The EEG-based BCI shows potential for enhancing the functional recovery of patients with motor disorders. The findings demonstrate that feature extraction methods PSD and DWT could effectively distinguish between periods of rest and activity in motor execution data. Nevertheless, for the intended application, including real-time processing based on single trial motor imagery data, BCI performance should be improved. Future research should focus on motor imagery EEG data encompassing motor imagery training and feedback on motor imagery performance.
...
Objective: To obtain a robust control signal the study 1) investigated the relation between event-related desynchronization (ERD) and mechanical stretch reflex size in the flexor carpi radialis across four muscle pre-loads consisting of 0%, 5%, 25% and 40% of maximum voluntary contraction (MVC), 2) investigated the ability of three offline signal processing paradigms in distinguishing between periods of rest and activity using EEG data associated with motor execution and motor imagery, 3) built a pseudo-online signal processing paradigm to simulate real-time signal processing based on a single trial and a continuous data stream.
Method: Mechanical stretch perturbations were applied to the wrist under four percentages of MVC during motor execution and imagery conditions in six healthy subjects. The data anal- ysis encompassed signal processing techniques including pre- processing with a large Laplacian filter, feature extraction through autoregressive modelling (AR), power spectral density (PSD), or discrete wavelet transform (DWT), and classification using linear discriminant analysis (LDA).
Results: Mechanical stretch reflex sizes and ERD amplitude significantly increased with increasing percentage of MVC for motor execution trials. For motor imagery trials, no significant correlation was found between the stretch reflex size and ERD amplitude. The offline signal processing paradigms resulted in classification accuracies of 73.55% (PSD), 71.96% (DWT) and 57.13% (AR). The classification accuracies significantly increased with increasing percentage of MVC. The pseudo-online paradigm resulted in a mean classification accuracy of 51.38%.
Conclusions: The EEG-based BCI shows potential for enhancing the functional recovery of patients with motor disorders. The findings demonstrate that feature extraction methods PSD and DWT could effectively distinguish between periods of rest and activity in motor execution data. Nevertheless, for the intended application, including real-time processing based on single trial motor imagery data, BCI performance should be improved. Future research should focus on motor imagery EEG data encompassing motor imagery training and feedback on motor imagery performance.
...
Background: A brain-computer interface (BCI) is a system that enables humans to control a computer by their brain signals. This can be achieved by modulating the sensorimotor rhythm (SMR) through both motor execution and motor imagery. The potential enhancement of spinal reflexes and motor control through SMR training is attributed to the hypothesis that activity-dependent brain plasticity guides spinal plasticity during motor skill learning. However, the signal processing needed for conversion from raw brain signals to a robust control signal is challenging. The recorded electroencephalogram (EEG) signals are contaminated by multiple unknown sources and suffer from inter-subject variability, complicating the development of the BCI.
Objective: To obtain a robust control signal the study 1) investigated the relation between event-related desynchronization (ERD) and mechanical stretch reflex size in the flexor carpi radialis across four muscle pre-loads consisting of 0%, 5%, 25% and 40% of maximum voluntary contraction (MVC), 2) investigated the ability of three offline signal processing paradigms in distinguishing between periods of rest and activity using EEG data associated with motor execution and motor imagery, 3) built a pseudo-online signal processing paradigm to simulate real-time signal processing based on a single trial and a continuous data stream.
Method: Mechanical stretch perturbations were applied to the wrist under four percentages of MVC during motor execution and imagery conditions in six healthy subjects. The data anal- ysis encompassed signal processing techniques including pre- processing with a large Laplacian filter, feature extraction through autoregressive modelling (AR), power spectral density (PSD), or discrete wavelet transform (DWT), and classification using linear discriminant analysis (LDA).
Results: Mechanical stretch reflex sizes and ERD amplitude significantly increased with increasing percentage of MVC for motor execution trials. For motor imagery trials, no significant correlation was found between the stretch reflex size and ERD amplitude. The offline signal processing paradigms resulted in classification accuracies of 73.55% (PSD), 71.96% (DWT) and 57.13% (AR). The classification accuracies significantly increased with increasing percentage of MVC. The pseudo-online paradigm resulted in a mean classification accuracy of 51.38%.
Conclusions: The EEG-based BCI shows potential for enhancing the functional recovery of patients with motor disorders. The findings demonstrate that feature extraction methods PSD and DWT could effectively distinguish between periods of rest and activity in motor execution data. Nevertheless, for the intended application, including real-time processing based on single trial motor imagery data, BCI performance should be improved. Future research should focus on motor imagery EEG data encompassing motor imagery training and feedback on motor imagery performance.
Objective: To obtain a robust control signal the study 1) investigated the relation between event-related desynchronization (ERD) and mechanical stretch reflex size in the flexor carpi radialis across four muscle pre-loads consisting of 0%, 5%, 25% and 40% of maximum voluntary contraction (MVC), 2) investigated the ability of three offline signal processing paradigms in distinguishing between periods of rest and activity using EEG data associated with motor execution and motor imagery, 3) built a pseudo-online signal processing paradigm to simulate real-time signal processing based on a single trial and a continuous data stream.
Method: Mechanical stretch perturbations were applied to the wrist under four percentages of MVC during motor execution and imagery conditions in six healthy subjects. The data anal- ysis encompassed signal processing techniques including pre- processing with a large Laplacian filter, feature extraction through autoregressive modelling (AR), power spectral density (PSD), or discrete wavelet transform (DWT), and classification using linear discriminant analysis (LDA).
Results: Mechanical stretch reflex sizes and ERD amplitude significantly increased with increasing percentage of MVC for motor execution trials. For motor imagery trials, no significant correlation was found between the stretch reflex size and ERD amplitude. The offline signal processing paradigms resulted in classification accuracies of 73.55% (PSD), 71.96% (DWT) and 57.13% (AR). The classification accuracies significantly increased with increasing percentage of MVC. The pseudo-online paradigm resulted in a mean classification accuracy of 51.38%.
Conclusions: The EEG-based BCI shows potential for enhancing the functional recovery of patients with motor disorders. The findings demonstrate that feature extraction methods PSD and DWT could effectively distinguish between periods of rest and activity in motor execution data. Nevertheless, for the intended application, including real-time processing based on single trial motor imagery data, BCI performance should be improved. Future research should focus on motor imagery EEG data encompassing motor imagery training and feedback on motor imagery performance.
Master thesis
(2022)
-
L. Thirunavukkarasu Kumaaravelu, J.C.F. de Winter, D. Dodou, V. Onkhar, P. Bazilinskyy, W. Tabone, Y.B. Eisma
Vulnerable road users account for more than 50% of traffic fatalities, and among these, pedestrians are the most susceptible to fatalities due to their distraction and misperception of other road users. To mitigate their plight, systems that warn drivers and pedestrians in case of a possible collision have been developed. Among systems that focus on the pedestrian's perspective, existing concepts are capable of predicting collisions but lack elements that monitor the visual attention of pedestrians. We address this gap by developing a gaze-based pedestrian warning system based on the Tobii Pro Glasses 2, a head-mounted eye-tracker. The system consists of: (1) a custom trained fast neural network (YOLO v4) on the KITTI object detection dataset that processes the video feed of the eye-tracker to detect approaching vehicles and (2) a module that uses the pedestrian's gaze to identify whether their attention falls on the closest moving vehicle that is approaching the pedestrian, both in real-time. If the pedestrian does not look at the approaching vehicle, they are given an auditory alert that warns them of a possible collision. In a pilot study conducted on a busy road in an urban environment, the system was evaluated under different pedestrian walking speeds and gaze behaviours to test the algorithm's robustness. The pilot study revealed that our system alerted the inattentive pedestrian with an accuracy of 67%. The mean vehicle detection accuracy and a mean moving vehicle identification accuracy from the pilot were 93% and 60%, respectively, a promising result given the use of only a mono camera. Despite the use of computer vision techniques, the system worked at an inference speed of 50 FPS due to the multi-processing capabilities of our algorithm. Our efforts are a first step in developing pedestrian warning systems based on eye-tracking technology to improve road safety in the future. The algorithm (Python-based) code used for this work has been made publicly available.
...
Vulnerable road users account for more than 50% of traffic fatalities, and among these, pedestrians are the most susceptible to fatalities due to their distraction and misperception of other road users. To mitigate their plight, systems that warn drivers and pedestrians in case of a possible collision have been developed. Among systems that focus on the pedestrian's perspective, existing concepts are capable of predicting collisions but lack elements that monitor the visual attention of pedestrians. We address this gap by developing a gaze-based pedestrian warning system based on the Tobii Pro Glasses 2, a head-mounted eye-tracker. The system consists of: (1) a custom trained fast neural network (YOLO v4) on the KITTI object detection dataset that processes the video feed of the eye-tracker to detect approaching vehicles and (2) a module that uses the pedestrian's gaze to identify whether their attention falls on the closest moving vehicle that is approaching the pedestrian, both in real-time. If the pedestrian does not look at the approaching vehicle, they are given an auditory alert that warns them of a possible collision. In a pilot study conducted on a busy road in an urban environment, the system was evaluated under different pedestrian walking speeds and gaze behaviours to test the algorithm's robustness. The pilot study revealed that our system alerted the inattentive pedestrian with an accuracy of 67%. The mean vehicle detection accuracy and a mean moving vehicle identification accuracy from the pilot were 93% and 60%, respectively, a promising result given the use of only a mono camera. Despite the use of computer vision techniques, the system worked at an inference speed of 50 FPS due to the multi-processing capabilities of our algorithm. Our efforts are a first step in developing pedestrian warning systems based on eye-tracking technology to improve road safety in the future. The algorithm (Python-based) code used for this work has been made publicly available.
Automated Guide-Path Map Generation for Automated Guided Vehicle Systems
An Algorithmic Framework
The design of a guide-path map is a well-known problem in various industries, such as automated guided vehicle (AGV) systems, road networks, and train track design. Nowadays, designing a path map is a manual, time-consuming, and expensive process. The solutions often turn out to be far from optimal. Several approaches are known to generate path maps automatically. The main challenges are computation efficiency due to large solution spaces and the lack of feasibility of solutions for application to real-world layouts. Little research focuses on the importance of computational efficiency to keep up with the pace of dynamic user requirements and constraints.
The goal is to design a framework that balances computational efficiency with feasibility in automated path map generation. A feasible path map can continuously distribute requested flow targets and is practically useable without many manual changes. It is expected that less optimal but still feasible solutions can be generated more efficiently by using a smaller solution space.
The framework consists of four stages: a practical problem, a mathematical model, a mathematical solution, and a practical solution. All solutions are theoretically validated on two aspects: capacity and performance.
The framework is tested using three layouts: low complexity and small-scale, complex and large scale, and one in between. The exploratory model outperforms in the complex large-scale layout while the goal-oriented models outperform in the other layouts.
Path maps are generated in a reasonable and controllable time. The settings allow a trade-off between the efficiency and feasibility of solutions. Due to its efficient computation, the model has the potential to generate path maps that are directly usable in real applications, if several realistic implementations are included to improve the feasibility of solutions.
...
The goal is to design a framework that balances computational efficiency with feasibility in automated path map generation. A feasible path map can continuously distribute requested flow targets and is practically useable without many manual changes. It is expected that less optimal but still feasible solutions can be generated more efficiently by using a smaller solution space.
The framework consists of four stages: a practical problem, a mathematical model, a mathematical solution, and a practical solution. All solutions are theoretically validated on two aspects: capacity and performance.
The framework is tested using three layouts: low complexity and small-scale, complex and large scale, and one in between. The exploratory model outperforms in the complex large-scale layout while the goal-oriented models outperform in the other layouts.
Path maps are generated in a reasonable and controllable time. The settings allow a trade-off between the efficiency and feasibility of solutions. Due to its efficient computation, the model has the potential to generate path maps that are directly usable in real applications, if several realistic implementations are included to improve the feasibility of solutions.
...
The design of a guide-path map is a well-known problem in various industries, such as automated guided vehicle (AGV) systems, road networks, and train track design. Nowadays, designing a path map is a manual, time-consuming, and expensive process. The solutions often turn out to be far from optimal. Several approaches are known to generate path maps automatically. The main challenges are computation efficiency due to large solution spaces and the lack of feasibility of solutions for application to real-world layouts. Little research focuses on the importance of computational efficiency to keep up with the pace of dynamic user requirements and constraints.
The goal is to design a framework that balances computational efficiency with feasibility in automated path map generation. A feasible path map can continuously distribute requested flow targets and is practically useable without many manual changes. It is expected that less optimal but still feasible solutions can be generated more efficiently by using a smaller solution space.
The framework consists of four stages: a practical problem, a mathematical model, a mathematical solution, and a practical solution. All solutions are theoretically validated on two aspects: capacity and performance.
The framework is tested using three layouts: low complexity and small-scale, complex and large scale, and one in between. The exploratory model outperforms in the complex large-scale layout while the goal-oriented models outperform in the other layouts.
Path maps are generated in a reasonable and controllable time. The settings allow a trade-off between the efficiency and feasibility of solutions. Due to its efficient computation, the model has the potential to generate path maps that are directly usable in real applications, if several realistic implementations are included to improve the feasibility of solutions.
The goal is to design a framework that balances computational efficiency with feasibility in automated path map generation. A feasible path map can continuously distribute requested flow targets and is practically useable without many manual changes. It is expected that less optimal but still feasible solutions can be generated more efficiently by using a smaller solution space.
The framework consists of four stages: a practical problem, a mathematical model, a mathematical solution, and a practical solution. All solutions are theoretically validated on two aspects: capacity and performance.
The framework is tested using three layouts: low complexity and small-scale, complex and large scale, and one in between. The exploratory model outperforms in the complex large-scale layout while the goal-oriented models outperform in the other layouts.
Path maps are generated in a reasonable and controllable time. The settings allow a trade-off between the efficiency and feasibility of solutions. Due to its efficient computation, the model has the potential to generate path maps that are directly usable in real applications, if several realistic implementations are included to improve the feasibility of solutions.