Circular Image

J.H.G. Dauwels

info

Please Note

56 records found

Journal article (2026) - P. Benschop, J.C. van Gemert, J.P. Mense, J.H.G. Dauwels
Video captured for action recognition often contains sensitive appearance cues such as faces, skin color, and clothing. Models trained on such data may exploit these cues rather than the underlying motion, raising privacy concerns in real-world deployment. In this work, we study action recognition under a motion-focused constraint: the model receives only motion representations that capture pixel displacement over time, while reducing appearance cues that expose identity or scene context. We focus on motion-history images and optical flow as learning-free representations that reduce identifiable appearance information while retaining action recognition accuracy. Our motion I3D model achieves approximately 31% and 52% zero-shot top-1 accuracy on HMDB-51 and UCF-101, respectively, outperforming non-CLIP direct-transfer baselines trained on Kinetics-400 despite operating without any appearance input. In 16-shot adaptation, the same model reaches 52% and 83% top-1 accuracy. In the domain adaptation setting on TP-HMDB↔TP-UCF, our motion-focused models achieve higher action recognition accuracy than prior privacy-preserving methods. Sensitive attribute predictability is reduced relative to RGB by a comparable margin, without requiring a learned privacy filter. On PA-HMDB51, optical flow is the strongest motion representation for privacy preservation, approaching chance level for skin-color prediction and remaining below RGB on most privacy attributes, indicating that motion representations retain useful action information while exposing less personal information. ...

Exploring the Potential of Linking Digital Product Passport Data, X-Ray Scanned Images, and AI

Conference paper (2026) - B.D. Rukanova, J.H.G. Dauwels, Ger C. M. Koomen, Y. Tan, Susana Wong Chan, Frank Janssens, Toni Männistö
Cross-border e-commerce is continuously growing with rapid speed which poses issues for authorities to monitor and control the large volumes of goods entering the EU via postal and express services. Digital Product Passports (DPP) are seen as a digital tool that can enable the e-commerce monitoring, however what roles DPPs can play is not yet fully understood. In this research, based on real-life piloting with scanned x-ray images of 10 packages with textiles and toys and based on product data, we gained insights and defined further research directions for exploring further the potential of DPPs and AI and scanned images for customs risk management in the context of cross-border e-commerce. ...

A machine learning approach using intraoperative neuromonitoring data

Journal article (2026) - Tamir Themans, Valerie Ter Wengel, Saskia van der Gaag, Justin Dauwels, Mark van de Ruit, Niels van der Gaag
Background Intraoperative neuromonitoring (IONM) reduces postoperative neurological complications, but its precise value for neurological outcomes remains unclear. Machine learning offers a fast, objective, real-time approach to analyzing large IONM datasets. We developed Machine learning models combining baseline characteristics and IONM data—using motor evoked potentials (MEPs) and somatosensory evoked potentials (SSEPs), separately and combined—to predict postoperative neurological outcomes and identify key predictive features. Methods In this retrospective cohort study, 67 patients undergoing spinal surgery (2019–2023) at Haga Teaching Hospital with sufficient IONM and clinical data were analyzed. Medical records and 260 IONM features were assessed; neurological status at 3 months postoperatively was categorized in classes relative to preoperative status as “stable deficits”, “intact”, or “improvement”. Using nested cross-validation, 4 classifiers—support vector machine, K-nearest neighbors, random forest, and extreme gradient boosting—were tested in addition to clinical data across MEP, SSEP, and combined modalities. Performance was expressed as sensitivity, specificity, accuracy, and precision. Results Extreme gradient boosting outperformed all classifiers on every metric. The combined MEP–SSEP model achieved the highest sensitivity (70.4%), specificity (88.3%), accuracy (87.1%), and best per-class scores, while the MEP model achieved the highest precision (75.6%). Key predictive features were preoperative neurological deficits (29%) and last intraoperative signal latency versus baseline (13.5%). Conclusions MEP and SSEP IONM features enhance prediction of 3-months neurological outcomes, provided preoperative status is accurately documented and incorporated. MEP features show superior predictive values compared to SSEP features when both modalities are accessible, with intraoperative signal latency change emerging as a prominent predictive IONM feature. ...
Physics-informed neural networks (PINNs) on complex domains are limited by input representations that encode geometry but do not reflect the physics of the governing PDE. We propose an operator-aware PINN for solid mechanics problems that embeds precomputed eigenmodes of the problem’s own discrete operator as geometry and physics-aware features within a weak-form variational formulation. The displacement field is represented by a neural mapping enriched by operator-aligned features rather than being constrained to a finite element trial space. A hybrid chain-rule formulation propagates spatial derivatives through both coordinate and eigenmode branches, yielding strain fields consistent with operator structure while preserving neural expressivity. The approach maintains the stability of variational formulations while mitigating the spectral bias and geometry-encoding limitations of classical PINNs. For inverse elasticity, the same eigenmodes act as virtual test functions in a weak-form residual, mitigating the stiffness-collapse pathology. The framework provides an instance-based alternative to differentiable finite element method without labeled training data. It achieves competitive accuracy and connects spectral operator theory with physics-informed neural computation. ...
Journal article (2026) - Davide Casnici, Charlotte Frenkel, Justin Dauwels
Backpropagation (BP) of errors is currently the backbone training algorithm for artificial neural networks (ANNs). It works by updating the network weights through gradient descent to minimize the value of a loss function, which represents the mismatch between the network’s prediction and the desired output. BP relies on the chain rule from calculus to propagate the loss gradient backward through the network’s hierarchy, allowing each weight to be efficiently and precisely updated based on its contribution to the output error. However, this process constrains the weight updates at every layer to rely on a global error signal generated at the extreme of the hierarchy. By contrast, the Hebbian model of synaptic plasticity in the brain states that weight updates should be local, determined only by the activity of presynaptic and postsynaptic neurons. According to Hebb’s model, it is therefore unlikely that biological brains directly implement BP. Recently, an alternative algorithm for training ANNs called predictive coding (PC) is gaining interest, appearing as a more biologically plausible alternative that updates the network weights using only local information. Originating from Elias’s 1950s work on signal compression [1], PC was later proposed in neuroscience as a model of the visual cortex by Rao and Ballard [2]. Successively, Friston formalized it under the free energy principle (FEP) [3], [4], grounding PC within the frameworks of Bayesian inference and dynamical systems. PC weight updates rely only on presynaptic and postsynaptic information, eliminating BP’s dependence on a global error signal. Moreover, it theoretically provides features beyond those of standard BP, such as the ability to automatically scale each gradient based on the associated uncertainty. Despite these advantages, PC still faces several open challenges: iterative error minimization can slow training compared to BP, and scaling to very deep architectures remains difficult [5]. At the same time, its local learning rules and modular structure offer promising opportunities, including highly parallelizable hardware implementations, biologically plausible learning in ANNs, and new connections between artificial intelligence (AI) and neuroscience. These aspects make PC an active area of research, with both theoretical and practical questions yet to be explored. This “Lecture Notes” column offers a novel, tutorial-style introduction to PC, focusing on its formulation, derivation, and connections to well-established optimization and signal processing algorithms, such as BP and the Kalman filter (KF). It aims to provide accessible support to the existing literature, guiding readers from the mathematical foundations underlying PC to its practical implementation, including computational examples in Python using the PyTorch framework. ...

An Explainable Approach

Conference paper (2025) - Christos Spiliadis, Yiheng Chang, Justin Dauwels, Chavdar Bachvarov, John J. Van Den Dobbelsteen, Benno H.W. Hendriks, Maarten Van Der Elst, Markku Eskola
Surgical workflow analysis optimizes efficiency, resource use, and patient safety in catheterization labs. Traditional manual methods are labour-intensive and inconsistent, driving the need for automated solutions that utilize machine learning and computer vision. This thesis introduces an explainable two-stage model for workflow analysis using ceiling-mounted cameras. The approach combines a YOLOv8 object detection model with a Gaussian Mixture Model - Hidden Markov Model (GMM-HMM). The first stage detects key objects for input into the second stage, where the GMM-HMM infers workflow phases by modelling spatial and temporal dynamics for real-time classification. Validation on two hospital datasets achieves 95.2% accuracy for the RdGG dataset and 95.4% for HH Tampere, demonstrating generalizability across environments. Experimental results show high accuracy in detecting workflow phases, highlighting explainability and robustness. The combined efficiencies of YOLOv8 and GMM-HMM allow for precise phase transition identification. The model's real-time application and adaptability across hospitals suggest its clinical implementation potential. This research furthers automated workflow analysis by enhancing interpretability and adaptability. Future work aims to improve robustness against occlusions, integrate audio data, and explore applications in other surgical settings. ...
Conference paper (2025) - S. Zhao, R. T. Rajan, A. N. Tallarico, M. Millesimo, V. Volosov, A. Imbruglia, J. Dauwels
The accurate prediction of Gallium Nitride High-Electron Mobility Transistors (GaN HEMTs) lifetime is essential for ensuring the reliability of power electronics. However, the complex and often competing degradation mechanisms within a single GaN-based transistor make lifetime extrapolation particularly challenging, especially under limited-data scenarios. In this work, we explore two machine learning approaches, i.e., XGBoost Regression and Gaussian Process Regression (GPR), for static gate lifetime prediction based on early measurements of current and ON-state resistance. In particular, we use features derived from empirical models to improve accuracy and model-specific methods to estimate uncertainty. We compare bootstrapped XGBoost ensembles, which yield empirical confidence intervals, with GPR, which provides analytical uncertainty estimates. Experiments on a time-dependent gate breakdown (TDGB) dataset spanning 16 voltage–temperature combinations show that GPR achieves an SMAPE of 8.8% and ECE of 0.028, outperforming XGBoost in Leave-One-Condition-Out Cross-Validation. These results highlight the feasibility of our proposed uncertainty-aware gate-lifetime prediction for Schottky p-GaN gate HEMTs in small-sample settings, and provide a basis for extending the framework towards time-dependent degradation modeling. ...
Journal article (2025) - Y. H.Victoria Chua, Justin Dauwels, Preman Rajalingam, Chew Lee Teo, Suzy J. Styles
Background: Intra-group discussions during actual TBL sessions play a huge role in knowledge consolidation and learning but are often understudied. Aims: Using a pre-registered study framework, we examined if participation equity (H1), reciprocal interaction (H2), information density (H3), mutual understanding (H4), and emotional rapport (H5) affected how much students learn from their intra-group team-based learning discussions and how they rated their team's discussions. Sample: Participants were 165 undergraduate students assigned to 28 teams. Methods: Using linguistic, conversational, and socio-affective features extracted from recordings of Year 1 and 2 medical students engaging in team-based learning, each construct was conceptualised at the level of the group and the individual. We used linear mixed-effects models and competing models approach to establish which of our metrics best account for the observed variance in individual learning gains and perceived collaboration quality. The analysis plan was preregistered, including correction for multiple comparisons. Results: None of our individual-level or group-level metrics significantly predicted individual learning gains. One of the group-level metrics significantly predicted perceived collaboration quality: reciprocal interaction. Our exploratory analysis found that individual baseline score of the best performer in the team positively predicted individual learning gains for others in their team, regardless of other interaction metrics. Conclusion: While students perceived the highest collaboration quality when turn-taking in their team was evenly distributed, the strongest predicter of learning gains for a student was the knowledge level of their top-scoring team-mate. This finding has implications for classroom equity, group formation and activity planning. ...

Techniques for Lane Marker Detection and Rain Removal

Book chapter (2025) - Sarat Chandra Nagavarapu, Anuj Abraham, Sihao Li, Justin Dauwels
Autonomous Vehicles (AVs) equipped with camera systems have emerged as a pivotal solution for smart urban mobility. The escalating demand for AVs emphasizes the need to prioritize driving safety, especially in challenging weather conditions like heavy rain. In this context, the accurate perception of environmental features, notably lane markers, becomes imperative for effective autonomous navigation. Severe weather can lead to camera image degradation, including blur and loss of details, impacting the accuracy of subsequent image processing. Despite the prevalence of camera-based methods, sensitivity to environmental noise, such as rain streaks, poses a challenge, necessitating preprocessing mechanisms like rain removal to enhance lane detection accuracy. This chapter focuses on the development of a vision-based algorithm dedicated to detecting and tracking lane markers, coupled with an efficient rain streak removal algorithm. A progressive approach to lane detection on city roads is presented, incorporating sliding windows and Kalman filter methodologies into a model-based method. Integration of the Kalman filter has yielded a notable improvement in video processing speeds, from 1.67 to 2.72 frames/s, enhancing overall operational efficiency. Furthermore, a novel neural network structure, amalgamating convolutional neural networks (CNNs) and long short-term memory (LSTM), is introduced for rain streak removal before performing lane marker detection. Comparative analysis against existing methods demonstrates an average 2.3% improvement in peak signal-to-noise ratio (PSNR) for rain removal and an 8% enhancement in Google Vision test results. ...
Conference paper (2025) - S. Li, R. T. Rajan, E. Marth, P. Zorn, W. Gruber, J. Dauwels
Intelligent Fault Detection (IFD) has garnered significant attention, with recent advances in AI-empowered predictive maintenance. A key challenge in applying IFD models lies in the interpretability of the methods, since the mechanisms are typically complex and difficult to integrate with data-driven approaches. In addition, the integration of edge devices is an emerging trend, which ensures fault detection and subsequent decision making on the edge, and thus offering an instant response as compared to a conventional centralized server-based architecture. However, to realize Edge-based IFD the primary constraints are low storage capacity and limited computational resources. In this paper, we address various critical challenges in automatic Edge-based IFD for motors in industrial settings, focusing on three key constraints, i.e., (a) limited availability of training data, (b) the lack of method interpretability, and (c) the computational and storage limitations of edge devices. To overcome these challenges, we propose a suite of light weight Physics-Informed (PI) AI algorithms to achieve Edge-based IFD - without compromising detection performance. We validate our proposed methods on experimental data for motor fault detection, and additionally present results from the implementation of these methods on an edge device. We discuss the benefits of our proposed solutions, and give directions for future work. ...

The 2nd Workshop on Large Generative Models Meet Multimodal Applications

Conference paper (2024) - Shihao Xu, Yiyang Luo, Justin Dauwels, Andy Khong, Zheng Wang, Qianqian Chen, Chen Cai, Wei Shi, Tat Seng Chua
This workshop aims to explore the potential of large generative models to revolutionize how we interact with multimodal information. A Large Language Model (LLM) represents a sophisticated form of artificial intelligence engineered to comprehend and produce natural language text, exemplified by technologies such as GPT, LLaMA, Flan-T5, ChatGLM, Qwen, etc. These models undergo training on extensive text datasets, exhibiting commendable attributes including robust language generation, zero-shot transfer capabilities, and In-Context Learning (ICL). With the surge in multimodal content-encompassing images, videos, audio, and 3D models-over the recent period, Large MultiModal Models (LMMs) have seen significant enhancements. These improvements enable the augmentation of conventional LLMs to accommodate multimodal inputs or outputs, as seen in BLIP, Flamingo, KOSMOS, LLaVA, Gemini, GPT-4, etc. Concurrently, certain research initiatives have developed specific modalities, with Kosmos2 and MiniGPT-5 focusing on image generation, and SpeechGPT on speech production. There are also endeavors to integrate LLMs with external tools to achieve a near "any-to-any" multimodal comprehension and generation capacity, illustrated by projects like Visual-ChatGPT, ViperGPT, MMREACT, HuggingGPT, and AudioGPT. Collectively,these models, spanning not only text and image generation but also other modalities, are referred to as large generative models. This workshop will allow researchers, practitioners, and industry professionals to explore the latest trends and best practices in the multimodal applications of large generative models. ...
Nowcasting leverages real-time atmospheric conditions to forecast weather over short periods. State-of-the-art models, including PySTEPS, encounter difficulties in accurately forecasting extreme weather events because of their unpredictable distribution patterns. In this study, we design a physics-informed neural network to perform precipitation nowcasting using the precipitation and meteorological data from the Royal Netherlands Meteorological Institute (KNMI). This model draws inspiration from the novel Physics-Informed Discriminator GAN (PID-GAN) formulation, directly integrating physics-based supervision within the adversarial learning framework. The proposed model adopts a GAN structure, featuring a Vector Quantization Generative Adversarial Network (VQ-GAN) and a Transformer as the generator, with a temporal discriminator serving as the discriminator. Our findings demonstrate that the PID-GAN model outperforms numerical and SOTA deep generative models in terms of precipitation nowcasting downstream metrics. ...
Journal article (2024) - Benjamin S. Grandey, Justin Dauwels, Zhi Yang Koh, Benjamin P. Horton, Lock Yue Chew
A probabilistic projection of sea-level rise uses a probability distribution to represent scientific uncertainty. However, alternative probabilistic projections of sea-level rise differ markedly, revealing ambiguity, which poses a challenge to scientific assessment and decision-making. To address the challenge of ambiguity, we propose a new approach to quantify a best estimate of the scientific uncertainty associated with sea-level rise. Our proposed fusion combines the complementary strengths of the ice sheet models and expert elicitations that were used in the Sixth Assessment Report (AR6) of the Intergovernmental Panel on Climate Change (IPCC). Under a low-emissions scenario, the fusion's very likely range (5th–95th percentiles) of global mean sea-level rise is 0.3–1.0 m by 2100. Under a high-emissions scenario, the very likely range is 0.5–1.9 m. The 95th percentile projection of 1.9 m can inform a high-end storyline, supporting decision-making for activities with low uncertainty tolerance. By quantifying a best estimate of scientific uncertainty, the fusion caters to diverse users. ...

A DNN Framework Expediting Content-based Video Image Retrieval (CBVIR)

Conference paper (2024) - Sinian Li, Doruk Barokas Profeta, Justin Dauwels
With the exponential growth of video data, individuals, particularly scholars in the fields of history and sociology, are increasingly reliant on video materials. However, the task of locating specific frames within videos remains a laborious and time-consuming endeavor. Advanced machine learning-assisted video processing techniques have emerged, including text-based video searches, video summarization, real-time object detection, and person re-identification. However, distinct from these, the main challenge of retrieving video frames based on given visual content is how to efficiently and accurately pinpoint the instance occurrences. To expedite the process while maintaining retrieval performance, we propose a two-stage approach, combining KeyFrame Extraction (KFE) and Content-based Image Retrieval (CBIR), underpinned a DNN-empowered framework called MoReSo. Our innovations include 1) the integration of improved statistical features with dynamic clustering in the KFE stage and 2) the development of the MoReSo framework, which consists of MobileNet and ResNet backbones with SOA layer to jointly represent video frames, achieving 2.67x increase in efficiency compared to existing solutions. Our framework is evaluated on two datasets: the annotated EHM Historical Database provided by digital history researchers and the widely-used image retrieval benchmark datasets, the Oxford and Paris datasets. The experimental results showcase that the proposed framework and scheme excel among other models in the CBVIR task. We make our code available for further exploration through our GitHub repository. This repository contains the implementation of our model and CBVIR system with a GUI prototype. ...
Conference paper (2024) - Shihao Xu, Yiyang Luo, Justin Dauwels, Andy Khong, Zheng Wang, Qianqian Chen, Chen Cai, Wei Shi, Tat Seng Chua
Journal article (2024) - Zhi Yang Koh, Benjamin S. Grandey, Dhrubajyoti Samanta, Adam D. Switzer, Benjamin P. Horton, J.H.G. Dauwels, Lock Yue Chew
Tide–surge interaction plays a substantial role in determining the characteristics of coastal water levels over shallow regions. We study the tide–surge interaction observed at seven tide gauges along Singapore and the east coast of Peninsular Malaysia, focusing on the timing of extreme non-tidal residuals relative to tidal high water. We propose a modified statistical framework using a no-tide–surge interaction (no-TSI) null distribution that accounts for asymmetry and variation in the duration of tidal cycles. We find that our modified framework can mitigate false-positive signals of tide–surge interaction in this region. We find evidence of tide–surge interaction at all seven locations, with characteristics varying smoothly along the coastline: the highest non-tidal residuals are found to occur most frequently before tidal high water in the south, both before and after tidal high water in the central region, and after tidal high water in the north. We also propose a semi-empirical model to investigate the effects of tidal-phase alteration, which is one mechanism of tide–surge interaction. Results of our semi-empirical model reveal that tidal-phase alteration caused by storm surges is substantial enough to generate significant change in the timing of extreme non-tidal residuals. To mitigate the effect of tidal-phase alteration on return level estimation, skew surge can be used. We conclude that (1) tide–surge interaction influences coastal water levels in this region, (2) our semi-empirical model provides insight into the mechanism of tidal-phase alteration, and (3) our no-TSI distribution should be used for similar studies globally. ...
Conference paper (2024) - Alan Hamo, Niki Ottenhof, Jan Wiebe H. Korstanje, Justin Dauwels
Cardiac output (CO) is a vital hemodynamic parameter that reflects the blood volume pumped by the heart per minute. A less-invasive way to estimate CO is by analyzing arterial blood pressure (ABP) waveforms. However, the relationship between CO and blood pressure is unknown. This study uses machine learning and feature engineering techniques to discover the relationship between CO and ABP. We apply the sparse identification non-linear dynamics (SINDy) algorithm to discover features. Additionally, we investigate the optimum number of cardiac cycles required for feature extraction to achieve the best performance. The proposed approach achieves clinically acceptable performance regarding radial limits of agreement (RLOA) and bias (RBias). Further, the proposed approach is validated on an external dataset. Finally, similarities to the Navier-Stokes equations are presented. ...
Deep learning-based object detectors, while offering exceptional performance, are data-dependent and can suffer from generalization issues. In this work, we investigated deep neural networks for detecting people and medical instruments for the vision-based workflow analysis system inside Catheterization Laboratories (Cath Labs). The central problem explored in this paper is the fact that the performance of the detector can degrade drastically if it is trained and tested on data from different Cath Labs. Our research aimed to investigate the underlying causes of this specific performance degradation and find solutions to mitigate this issue. We employed the YOLOv8 object detector and created datasets from clinical procedures recorded at Reinier de Graaf Hospital (RdGG) and Philips Best Campus, supplemented with publicly accessible images. Through a series of experiments complemented by data visualization, we discovered that the performance degradation primarily stems from data distribution shifts in the feature space. Notably, the object detector trained on non-sensitive online images can generalize to unseen Cath Labs, outperforming the model trained on a procedure recording from a different Cath Lab. The detector trained on the online images achieved an mAP@0.5 of 0.517 on the RdGG dataset. Furthermore, by switching to the most suitable camera for each object in the Cath Lab, the multi-camera system can further improve the detection performance significantly. An aggregated L-camera mAP@0.5 of 0.679 is achieved for single-object classes on the RdGG dataset. ...

A Hybrid Approach for Surface Mounted Electronics

Industrial assembly lines are the heartbeat of modern manufacturing, where precision and efficiency are paramount. This paper introduces a novel hybrid Explainable artificial intelligence (XAI) approach to enhance monitoring and analysis in industrial assembly. By fusing the power of vision anomaly detection models with the clarity of the gradient tree boosting algorithm, this framework not only boosts defect detection accuracy but also provides transparent, actionable insights. This synergy transforms how operators and engineers interact with AI, fostering trust and enhancing operational excellence. ...

From Single-Target to Brain-Network Modulation: Why and What Is Needed?

Objectives: Current techniques in brain stimulation are still largely based on a phrenologic approach that a single brain target can treat a brain disorder. Nevertheless, meta-analyses of brain implants indicate an overall success rate of 50% improvement in 50% of patients, irrespective of the brain-related disorder. Thus, there is still a large margin for improvement. The goal of this manuscript is to 1) develop a general theoretical framework of brain functioning that is amenable to surgical neuromodulation, and 2) describe the engineering requirements of the next generation of implantable brain stimulators that follow from this theoretic model. Materials and Methods: A neuroscience and engineering literature review was performed to develop a universal theoretical model of brain functioning and dysfunctioning amenable to surgical neuromodulation. Results: Even though a single target can modulate an entire network, research in network science reveals that many brain disorders are the consequence of maladaptive interactions among multiple networks rather than a single network. Consequently, targeting the main connector hubs of those multiple interacting networks involved in a brain disorder is theoretically more beneficial. We, thus, envision next-generation network implants that will rely on distributed, multisite neuromodulation targeting correlated and anticorrelated interacting brain networks, juxtaposing alternative implant configurations, and finally providing solid recommendations for the realization of such implants. In doing so, this study pinpoints the potential shortcomings of other similar efforts in the field, which somehow fall short of the requirements. Conclusion: The concept of network stimulation holds great promise as a universal approach for treating neurologic and psychiatric disorders. ...