Convolutional neural networks (CNNs) trained on RGB images (red, green, blue channels) often exhibit sharp performance degradation under distribution shifts, as they tend to rely on superficial appearance cues such as background or texture. While depth information is known to provide complementary geometric signals that can improve robustness, most existing approaches assume access to ground-truth depth or rely on complex RGB-D architectures, limiting their applicability in practice.

In this work, we investigate whether estimated depth, obtained from a monocular RGB image, can serve as a simple and effective auxiliary signal to improve out-of-distribution (OOD) generalisation in standard CNN classifiers. Using both controlled toy experiments and real-world evaluations on the NICO++ benchmark, we compare RGB-only models against RGB-D variants that incorporate a single predicted depth channel via minimal fusion. Our results show that pseudo-depth consistently reduces OOD performance gaps across multiple CNN backbones, without degrading in-distribution accuracy. We further demonstrate that these gains persist under moderate corruption of the depth signal and disappear when geometric structure is entirely removed, indicating that the improvements stem from meaningful geometric information rather than the mere presence of an additional input channel. Furthermore, we analyse these effects through class-resolved confusion matrices and qualitative input-level examples, showing that depth specifically attenuates structured semantic confusions under domain shift.

Taken together, our findings suggest that even imperfect, predicted depth can act as a lightweight geometric inductive bias, helping CNN classifiers move away from brittle appearance-based shortcuts and toward more robust representations under domain shift.

https://gitlab.ewi.tudelft.nl/in5000/janvangemert/alexandraioana

The circular restricted three-body problem is a canonical example of chaotic dynamics and forms the basis of many advanced spacecraft trajectory designs. This thesis investigates whether emerging artificial intelligence based generative and regression methods can reduce computational costs and enable new tools for exploring families of periodic orbits in mission design.

Generative models are evaluated for their ability to reconstruct, sample, and represent multiple periodic-orbit families and their bifurcation structure, while regression-based surrogates are assessed for unstable manifold propagation. A loss formulation that explicitly incorporates the Jacobi constant is introduced, encouraging approximate conservation of energy within the system, and penalizing in-sequence variations in Jacobi.Generative models (variational autoencoders, transformer-based diffusion models) successfully capture orbital structure and orbital family bifurcationary relationships and support efficient exploration, though differential correction is typically required to enforce physical validity. Regression-based surrogates (Kolmogorov-Arnold and deep neural networks) reproduce qualitative behaviour but remain insufficiently accurate for mission design.

Parameter-Efficient Fine-Tuning (PEFT) methods for Transformers are designed for floating-point weights. When applied to extremely low-bit models (e.g., ternary {-1,0,1) they convert the base weights to floating point (dequantization) to add the update and then quantize again, which can diminish the benefits of aggressive quantization. We introduce a multiplicative ternary adapter that enables in-domain fine-tuning by applying an element-wise ternary mask to the base ternary Transformer weights, avoiding any dequantization to floating point and allowing direct merging back into the model. Constructed as the Kronecker product of two small trainable matrices and applied via a Hadamard product, the adapter preserves the ternary domain and merges with zero inference overhead. On a ternarized Llama-3.2-1B model, our method recovers substantial accuracy and surpasses stronger 2-bit baselines on most tasks, while retaining the efficiency advantages of ternary weights.

Much progress in optical flow research has been driven by benchmark datasets. However, these datasets provide only limited feedback on the underlying causes of architectural failures, typically restricted to metrics such as end-point error (EPE), occlusion statistics, and large-displacement ranges. This leads to imprecise claims regarding areas consecutive models have improved upon. In this paper, we present an analysis tool that enables the generation of customisable datasets, allowing controlled variation in displacement size, camera corruptions, luminance, and other factors. We demonstrate the utility of this tool by analysing the behaviour of different architectures under varying displacement sizes and in low-light settings.

Evacuation slides are critical aircraft safety components governed by stringent regulatory standards set by agencies like the Federal Aviation Administration and European Union Aviation Safety Agency. To comply with these standards, the maintenance and repacking of slides, currently performed manually, require operators to follow hundreds of precise steps. This process is labor-intensive, error-prone, and can result in costly delays and safety risks due to human error. Real-time visual inspection systems can help address these challenges, however, a key obstacle for real-world deployment of such systems is the scarcity of benchmark data from aerospace factory operations needed to validate and verify their performance. To enable this, we introduce the first known dataset tailored for evacuation slide inspection, comprising over 14,500 images captured under real and controlled conditions. This data aims to capture slide folding procedures of Embraer AFT evacuation slides, such that developed real-time systems can: (1) estimate the occluded position of the Pressure Relief Valve (PRV), (2) detect context-sensitive foreign objects such as packing clamps, and (3) calculate slide fold dimensions to prevent tolerance stacking errors. From this dataset, five benchmarks were constructed to evaluate performance across the three requirements. Baseline models were developed, including a PRV localization network using LSTMs, a variational autoencoder and object detection pipeline for packing clamp FOD, and a depth and reference-based slide fold measurement calculation method. When tested on our benchmarks, the depth-based measurement estimator showed precision and accuracy, the clamp FOD methodology showed high precision for images taken from specific cameras, however, the PRV position estimation remains a challenge that requires further research. Overall, our results set a foundation for the automation of visual inspection in slide packing and offer benchmarks for future research in this safety-critical inspection task.

Human Action Recognition (HAR) models are increasingly deployed in high-stakes environments, yet their fairness across different human appearances has not been analyzed. We introduce a framework for auditing bias in HAR models using synthetic video data, generated with full control over visual identity attributes such as skin color. Unlike prior work that focuses on static images or pose estimation, our approach preserves temporal consistency, allowing us to isolate and test how changes to a single attribute affect model predictions. Through controlled interventions using the BEDLAM simulation platform, we show whether some popular HAR models exhibit statistically significant biases on the skin color even when the motion remains identical. Our results highlight how models may encode unwanted visual associations, and we provide evidence of systematic errors across groups. This work contributes a framework for auditing HAR models and supports the development of more transparent, accountable systems in light of upcoming regulatory standards.

Test-time adaptation methods assume privileged access to model internals: parameters for fine-tuning, statistics for recalibration, or architectural components for modification. This assumption fails when models are deployed as certified systems, encrypted services, or under regulatory constraints that prohibit parameter changes. We present ITEM (Input Transformation via Entropy Minimization), the first preparation-agnostic sample adaptation method for test-time adaptation. ITEM learns input transformations that minimize prediction entropy using only gradient signals through frozen models, exploiting the principle that well-calibrated models produce low-entropy outputs on familiar data. Unlike existing sample adaptation methods that require specialized training procedures or parameter updates, ITEM works with any pre-trained model without modification or preparation requirements. Using scalar transformations as proof of concept, we demonstrate adaptation under extreme data scarcity: models trained on 10 samples per class and adapted with single calibration samples. ITEM significantly reduces performance degradation from corruption while existing methods fail or show negligible improvement. Our results establish that effective test-time adaptation is possible without model modification, architectural knowledge, or training preparation, opening new possibilities for adapting deployed models under real-world constraints.

Distinguishing between benign and malignant ovarian cysts is a challenging task that depends on subjective visual markers in ultrasound scans. Current manual methods remain prone to costly misdiagnoses and the application of these methods depend heavily on the clinician's level of expertise. Recent research demonstrates promising applications of Convolutional Neural Networks (CNNs) for ovarian tumor classification; however, we observed that their performance is limited when applied to a diverse and complex dataset. To address this, we propose, implement, and evaluate three improvements to a baseline classifier.

First, we use a deep learning-based approach to remove burned-in medical annotations and introduce a weighted mean squared error (MSE) loss to improve its effectiveness by emphasizing relevant regions. This aims to better recover the original image content prior to annotation and remove annotations which can act as confounders. Second, we enhance classification by fusing image features with two readily available clinical factors at an intermediate stage of the network. Third, and central to this study, we incorporate a segmentation path that acts as a regularizer, encouraging the shared encoder to learn lesion-specific features that benefit the classification head.

These three contributions are informed by domain-specific knowledge of ovarian lesions and collectively demonstrate promising directions for improving deep learning-based models in this setting.

Spiking neural networks (SNNs) with Time-to-First-Spike (TTFS) coding promise rapid, sparse, and energy-efficient inference. However, the impact of sample difficulty on TTFS dynamics remains underexplored. We investigate (i) how input hardness influences first-spike timing and (ii) whether training on hard samples expedites inference. By quantifying difficulty via geometric margins and Gaussian-noise perturbations, and modeling leaky integrate-and-fire dynamics as Gaussian random walks, we derive first-hitting-time predictions. We further show that training-time noise, akin to ridge regularization, reduces weight variance and increases expected spike latencies. Empirical results on a synthetic task, MNIST, NMNIST, and CIFAR-10 with spiking MLPs/CNNs confirm that harder inputs slow inference and noise-trained models trade robustness for latency. Our findings align TTFS behavior with drift-diffusion models and provide a framework for balancing speed and robustness in neuromorphic SNNs.

Optical flow models excel on synthetic benchmarks but can struggle with real-world scenarios involving large displacements, which are critical for applications like autonomous navigation and augmented reality. To address this, we introduce a novel real-world dataset and evaluation framework, using a specialized annotation tool to capture ground truth optical flow in scenarios with fast movements and close-range objects. Our approach minimizes confounders, providing clear insights into model performance with large displacements. Findings show recent models outperform the previous state-of-the-art, RAFT, across all tested scenarios. Both the annotation tool and dataset are available to support further research.

Optical flow estimation models are currently trained and evaluated on synthetic datasets. However, the generalizability of these models to real-world applications remains unexplored. This study investigates how well two state-of-the-art optical flow estimation models perform on real-world Articulated, Homothetic, and Conformal non-rigid motion. To facilitate evaluation, a manually annotated dataset comprising twenty-four real-world image pairs and sparse vector fields was created. Both models demonstrated performance consistent with synthetic benchmarks on Homothetic and Conformal motion. However, results degraded when evaluating Articulated motion, revealing limitations in real-world applicability for practical applications such as controlled robotics and object tracking.

Occlusions are one of the main challenges in optical flow estimation, where parts of the scene are no longer visible between consecutive frames. Several models address this problem, either intrinsically or explicitly, using different strategies. However, most benchmarks rely on synthetic data, and even real-world ones evaluate only overall model performance, without isolating occlusions. This work investigates optical flow model performance under real-world occlusions by introducing a manually annotated, occlusion-focused dataset. We present an annotation method tailored to three occlusion types: out-of-frame, inter-object, and self-occlusion. We then evaluate two models, FlowFormer++ and CCMR, which handle occlusions using different mechanisms. Our findings show that while CCMR demonstrates stronger overall performance, both models struggle with occluded regions, particularly self-occlusions involving rotation and perspective transformations. These results highlight the need for improved occlusion reasoning in models and more diverse real-world benchmarks.

Optical flow estimation is a core task in computer vision, yet many existing models struggle with lighting-induced appearance changes that are common in real-world scenarios. This work presents a focused evaluation of recent deep learning-based optical flow models under controlled lighting variations, using a custom dataset composed of indoor and outdoor scenes recorded with a static camera. Scenarios include glare, moving shadows, intensity shifts, and outdoor shadows, with ground truth flow defined as zero to isolate the effect of illumination changes. Four models—RAFT, GMFlow, SEA-RAFT, and FlowDiffuser—are benchmarked using standard metrics (EPE and F1-all). The results reveal that even in the absence of physical motion, several models produce significant flow estimates, particularly under shadow and intensity variation. SEA-RAFT and RAFT show relatively higher robustness, while GMFlow and FlowDiffuser are more sensitive to lighting artifacts. The findings highlight a critical gap in current model generalization and emphasize the need for lighting-aware architectures and training strategies.

Egocentric video provides a unique view into human perception and interaction, with growing relevance for augmented reality, robotics, and assistive technologies. However, rapid camera motion and complex scene dynamics pose major challenges for 3D reconstruction from this perspective. While 3D Gaussian Splatting (3DGS) has become a state-of-the-art method for efficient, high-quality novel view synthesis, variants, that focus on reconstructing dynamic scenes from monocular video are rarely evaluated on egocentric video. It remains unclear whether existing models generalize to this setting or if egocentric-specific solutions are needed. In this work, we evaluate dynamic monocular 3DGS models on egocentric and exocentric video using paired ego-exo recordings from the EgoExo4D dataset. We find that reconstruction quality is consistently lower in egocentric views. Analysis reveals that the difference in reconstruction quality, measured in peak signal-to-noise ratio, stems from the reconstruction of static, not dynamic, content. Our findings underscore current limitations and motivate the development of egocentric-specific approaches, while also highlighting the value of separately evaluating static and dynamic regions of a video

Egocentric video provides a unique view into human perception and interaction, with growing relevance for augmented reality, robotics, and assistive technologies. However, rapid camera motion and complex scene dynamics pose major challenges for 3D reconstruction from this perspective. While 3D Gaussian Splatting (3DGS) has become a state-of-the-art method for efficient, high-quality novel view synthesis, variants, that focus on reconstructing dynamic scenes from monocular video are rarely evaluated on egocentric video. It remains unclear whether existing models generalize to this setting or if egocentric-specific solutions are needed. In this work, we evaluate dynamic monocular 3DGS models on egocentric and exocentric video using paired ego-exo recordings from the EgoExo4D dataset. We find that reconstruction quality is consistently lower in egocentric views. Analysis reveals that the difference in reconstruction quality, measured in peak signal-to-noise ratio, stems from the reconstruction of static, not dynamic, content. Our findings underscore current limitations and motivate the development of egocentric-specific approaches, while also highlighting the value of separately evaluating static and dynamic regions of a video.

Visual counting is an important task in computer vision with broad applications in areas such as crowd monitoring, agriculture, and environmental analysis. While deep learning has significantly advanced this field by enabling models to learn robust feature representations, deep learning approaches suffer from sensitivity to data imbalances, which occur in the distribution of object counts across counting datasets as a result of annotation effort. Most state-of-the-art counting models, categorized into clustering-, detection-, regression-, and density estimation-based methods, are built upon Convolutional Neural Networks (CNNs) and Transformers, both of which are known to be susceptible to imbalances in the training data. This study introduces a hybrid model that incorporates a programmatically guaranteed counting mechanism using the RASP language and the Tracr compiler, enabling the construction of Transformer-based models that can reliably execute predefined tasks, such as counting. By combining this exact counting mechanism with a trainable embedding module, we present a model that is capable of learning to count various tokens, even under significant data imbalance. We validate our approach on a synthetic, imbalanced dataset and compare its performance, training time, and data efficiency against standard CNN- and Transformer-based models. Results suggest that our method achieves strong generalization across the full spectrum of object counts while requiring less training data, highlighting the potential for this architecture to be further investigated and adapted to be used for robust and efficient visual counting.

Audio-to-motion generation is an important task with applications in virtual avatar creation for XR systems and intelligent robot control in daily life scenarios.
Most current motion generation methods depend on a single encoder-decoder architecture to simultaneously model all body parts, constraining their capacity to capture the diverse and complex motions exhibited by humans.
In this paper, we propose a novel method, SpeechCAT, that employs three separate encoder-decoder modules to individually model the motions of the face, body, and hands. To capture the relationships and synchronization among these body parts, we introduce a cross-attention mechanism to effectively learn their correlations.
SpeechCAT ensures sufficient capacity to model the unique characteristics of each body part while preserving the coherence between them.
Our experimental results demonstrate the superiority of SpeechCAT over baseline methods, highlighting its effectiveness in generating diverse, realistic, and synchronized motions with face, body, and hand parts.

Early detection of leprosy, a neglected tropical disease, is crucial to preventing irreversible nerve damage and disability. Analyzing temperature vari- ations in hands using infrared (IR) cameras offers a potential low-cost alternative to existing medical equipment for early detection of leprosy. This study explores the adaptation of hand landmark detec- tion models, commonly used for hand pose track- ing, to infer the hypothenar area, a critical region for leprosy diagnosis. The research addresses the challenge of limited ground-truth data for the hy- pothenar keypoint by developing annotated datasets and evaluating machine learning models like Lasso Regression and XGBoost. These models signif- icantly outperform the existing method of linear interpolation, demonstrating the feasibility of ac- curate hypothenar keypoint prediction even with limited training data. The findings contribute to the development of accessible, automated tools for early leprosy diagnosis, particularly in resource- constrained settings.

This study investigates sensor technologies for di- agnosing leprosy in Nepal, focussing on skin tem- perature in the hands using contact and non-contact sensors. Leprosy affects the peripheral nervous system, causing thermoregulatory dysfunction de- tectable via localized skin temperature changes. A systematised comparative review compares contact thermometry, infrared (IR) thermography, and IR- RGB thermography based on measurement quality, usability, and cost. Next to the systematised re- view, an experimental method is proposed to com- bine RGB and IR imaging to enhance the spatial accuracy of automatic region of interest (ROI) de- tection using MediaPipe Hand Landmarker. The study introduces a multimodal dataset of 45 sets of annotated IR-RGB images and validates a geomet- rical image registration model, achieving 93.2% keypoint detection accuracy—significantly outper- forming IR-only sensors. Results show IR-RGB thermography as a cost-effective, flexible, and ac- curate tool for early leprosy diagnosis in resource- limited settings.