Parameter-Efficient Fine-Tuning (PEFT) methods for Transformers are designed for floating-point weights. When applied to extremely low-bit models (e.g., ternary {-1,0,1) they convert the base weights to floating point (dequantization) to add the update and then quantize again, which can diminish the benefits of aggressive quantization. We introduce a multiplicative ternary adapter that enables in-domain fine-tuning by applying an element-wise ternary mask to the base ternary Transformer weights, avoiding any dequantization to floating point and allowing direct merging back into the model. Constructed as the Kronecker product of two small trainable matrices and applied via a Hadamard product, the adapter preserves the ternary domain and merges with zero inference overhead. On a ternarized Llama-3.2-1B model, our method recovers substantial accuracy and surpasses stronger 2-bit baselines on most tasks, while retaining the efficiency advantages of ternary weights.

Much progress in optical flow research has been driven by benchmark datasets. However, these datasets provide only limited feedback on the underlying causes of architectural failures, typically restricted to metrics such as end-point error (EPE), occlusion statistics, and large-displacement ranges. This leads to imprecise claims regarding areas consecutive models have improved upon. In this paper, we present an analysis tool that enables the generation of customisable datasets, allowing controlled variation in displacement size, camera corruptions, luminance, and other factors. We demonstrate the utility of this tool by analysing the behaviour of different architectures under varying displacement sizes and in low-light settings.

Evacuation slides are critical aircraft safety components governed by stringent regulatory standards set by agencies like the Federal Aviation Administration and European Union Aviation Safety Agency. To comply with these standards, the maintenance and repacking of slides, currently performed manually, require operators to follow hundreds of precise steps. This process is labor-intensive, error-prone, and can result in costly delays and safety risks due to human error. Real-time visual inspection systems can help address these challenges, however, a key obstacle for real-world deployment of such systems is the scarcity of benchmark data from aerospace factory operations needed to validate and verify their performance. To enable this, we introduce the first known dataset tailored for evacuation slide inspection, comprising over 14,500 images captured under real and controlled conditions. This data aims to capture slide folding procedures of Embraer AFT evacuation slides, such that developed real-time systems can: (1) estimate the occluded position of the Pressure Relief Valve (PRV), (2) detect context-sensitive foreign objects such as packing clamps, and (3) calculate slide fold dimensions to prevent tolerance stacking errors. From this dataset, five benchmarks were constructed to evaluate performance across the three requirements. Baseline models were developed, including a PRV localization network using LSTMs, a variational autoencoder and object detection pipeline for packing clamp FOD, and a depth and reference-based slide fold measurement calculation method. When tested on our benchmarks, the depth-based measurement estimator showed precision and accuracy, the clamp FOD methodology showed high precision for images taken from specific cameras, however, the PRV position estimation remains a challenge that requires further research. Overall, our results set a foundation for the automation of visual inspection in slide packing and offer benchmarks for future research in this safety-critical inspection task.

Human Action Recognition (HAR) models are increasingly deployed in high-stakes environments, yet their fairness across different human appearances has not been analyzed. We introduce a framework for auditing bias in HAR models using synthetic video data, generated with full control over visual identity attributes such as skin color. Unlike prior work that focuses on static images or pose estimation, our approach preserves temporal consistency, allowing us to isolate and test how changes to a single attribute affect model predictions. Through controlled interventions using the BEDLAM simulation platform, we show whether some popular HAR models exhibit statistically significant biases on the skin color even when the motion remains identical. Our results highlight how models may encode unwanted visual associations, and we provide evidence of systematic errors across groups. This work contributes a framework for auditing HAR models and supports the development of more transparent, accountable systems in light of upcoming regulatory standards.

Test-time adaptation methods assume privileged access to model internals: parameters for fine-tuning, statistics for recalibration, or architectural components for modification. This assumption fails when models are deployed as certified systems, encrypted services, or under regulatory constraints that prohibit parameter changes. We present ITEM (Input Transformation via Entropy Minimization), the first preparation-agnostic sample adaptation method for test-time adaptation. ITEM learns input transformations that minimize prediction entropy using only gradient signals through frozen models, exploiting the principle that well-calibrated models produce low-entropy outputs on familiar data. Unlike existing sample adaptation methods that require specialized training procedures or parameter updates, ITEM works with any pre-trained model without modification or preparation requirements. Using scalar transformations as proof of concept, we demonstrate adaptation under extreme data scarcity: models trained on 10 samples per class and adapted with single calibration samples. ITEM significantly reduces performance degradation from corruption while existing methods fail or show negligible improvement. Our results establish that effective test-time adaptation is possible without model modification, architectural knowledge, or training preparation, opening new possibilities for adapting deployed models under real-world constraints.

Distinguishing between benign and malignant ovarian cysts is a challenging task that depends on subjective visual markers in ultrasound scans. Current manual methods remain prone to costly misdiagnoses and the application of these methods depend heavily on the clinician's level of expertise. Recent research demonstrates promising applications of Convolutional Neural Networks (CNNs) for ovarian tumor classification; however, we observed that their performance is limited when applied to a diverse and complex dataset. To address this, we propose, implement, and evaluate three improvements to a baseline classifier.

First, we use a deep learning-based approach to remove burned-in medical annotations and introduce a weighted mean squared error (MSE) loss to improve its effectiveness by emphasizing relevant regions. This aims to better recover the original image content prior to annotation and remove annotations which can act as confounders. Second, we enhance classification by fusing image features with two readily available clinical factors at an intermediate stage of the network. Third, and central to this study, we incorporate a segmentation path that acts as a regularizer, encouraging the shared encoder to learn lesion-specific features that benefit the classification head.

These three contributions are informed by domain-specific knowledge of ovarian lesions and collectively demonstrate promising directions for improving deep learning-based models in this setting.

Spiking neural networks (SNNs) with Time-to-First-Spike (TTFS) coding promise rapid, sparse, and energy-efficient inference. However, the impact of sample difficulty on TTFS dynamics remains underexplored. We investigate (i) how input hardness influences first-spike timing and (ii) whether training on hard samples expedites inference. By quantifying difficulty via geometric margins and Gaussian-noise perturbations, and modeling leaky integrate-and-fire dynamics as Gaussian random walks, we derive first-hitting-time predictions. We further show that training-time noise, akin to ridge regularization, reduces weight variance and increases expected spike latencies. Empirical results on a synthetic task, MNIST, NMNIST, and CIFAR-10 with spiking MLPs/CNNs confirm that harder inputs slow inference and noise-trained models trade robustness for latency. Our findings align TTFS behavior with drift-diffusion models and provide a framework for balancing speed and robustness in neuromorphic SNNs.

Optical flow models excel on synthetic benchmarks but can struggle with real-world scenarios involving large displacements, which are critical for applications like autonomous navigation and augmented reality. To address this, we introduce a novel real-world dataset and evaluation framework, using a specialized annotation tool to capture ground truth optical flow in scenarios with fast movements and close-range objects. Our approach minimizes confounders, providing clear insights into model performance with large displacements. Findings show recent models outperform the previous state-of-the-art, RAFT, across all tested scenarios. Both the annotation tool and dataset are available to support further research.

Optical flow estimation models are currently trained and evaluated on synthetic datasets. However, the generalizability of these models to real-world applications remains unexplored. This study investigates how well two state-of-the-art optical flow estimation models perform on real-world Articulated, Homothetic, and Conformal non-rigid motion. To facilitate evaluation, a manually annotated dataset comprising twenty-four real-world image pairs and sparse vector fields was created. Both models demonstrated performance consistent with synthetic benchmarks on Homothetic and Conformal motion. However, results degraded when evaluating Articulated motion, revealing limitations in real-world applicability for practical applications such as controlled robotics and object tracking.

Optical flow estimation is a core task in computer vision, yet many existing models struggle with lighting-induced appearance changes that are common in real-world scenarios. This work presents a focused evaluation of recent deep learning-based optical flow models under controlled lighting variations, using a custom dataset composed of indoor and outdoor scenes recorded with a static camera. Scenarios include glare, moving shadows, intensity shifts, and outdoor shadows, with ground truth flow defined as zero to isolate the effect of illumination changes. Four models—RAFT, GMFlow, SEA-RAFT, and FlowDiffuser—are benchmarked using standard metrics (EPE and F1-all). The results reveal that even in the absence of physical motion, several models produce significant flow estimates, particularly under shadow and intensity variation. SEA-RAFT and RAFT show relatively higher robustness, while GMFlow and FlowDiffuser are more sensitive to lighting artifacts. The findings highlight a critical gap in current model generalization and emphasize the need for lighting-aware architectures and training strategies.

Occlusions are one of the main challenges in optical flow estimation, where parts of the scene are no longer visible between consecutive frames. Several models address this problem, either intrinsically or explicitly, using different strategies. However, most benchmarks rely on synthetic data, and even real-world ones evaluate only overall model performance, without isolating occlusions. This work investigates optical flow model performance under real-world occlusions by introducing a manually annotated, occlusion-focused dataset. We present an annotation method tailored to three occlusion types: out-of-frame, inter-object, and self-occlusion. We then evaluate two models, FlowFormer++ and CCMR, which handle occlusions using different mechanisms. Our findings show that while CCMR demonstrates stronger overall performance, both models struggle with occluded regions, particularly self-occlusions involving rotation and perspective transformations. These results highlight the need for improved occlusion reasoning in models and more diverse real-world benchmarks.

Egocentric video provides a unique view into human perception and interaction, with growing relevance for augmented reality, robotics, and assistive technologies. However, rapid camera motion and complex scene dynamics pose major challenges for 3D reconstruction from this perspective. While 3D Gaussian Splatting (3DGS) has become a state-of-the-art method for efficient, high-quality novel view synthesis, variants, that focus on reconstructing dynamic scenes from monocular video are rarely evaluated on egocentric video. It remains unclear whether existing models generalize to this setting or if egocentric-specific solutions are needed. In this work, we evaluate dynamic monocular 3DGS models on egocentric and exocentric video using paired ego-exo recordings from the EgoExo4D dataset. We find that reconstruction quality is consistently lower in egocentric views. Analysis reveals that the difference in reconstruction quality, measured in peak signal-to-noise ratio, stems from the reconstruction of static, not dynamic, content. Our findings underscore current limitations and motivate the development of egocentric-specific approaches, while also highlighting the value of separately evaluating static and dynamic regions of a video

Egocentric video provides a unique view into human perception and interaction, with growing relevance for augmented reality, robotics, and assistive technologies. However, rapid camera motion and complex scene dynamics pose major challenges for 3D reconstruction from this perspective. While 3D Gaussian Splatting (3DGS) has become a state-of-the-art method for efficient, high-quality novel view synthesis, variants, that focus on reconstructing dynamic scenes from monocular video are rarely evaluated on egocentric video. It remains unclear whether existing models generalize to this setting or if egocentric-specific solutions are needed. In this work, we evaluate dynamic monocular 3DGS models on egocentric and exocentric video using paired ego-exo recordings from the EgoExo4D dataset. We find that reconstruction quality is consistently lower in egocentric views. Analysis reveals that the difference in reconstruction quality, measured in peak signal-to-noise ratio, stems from the reconstruction of static, not dynamic, content. Our findings underscore current limitations and motivate the development of egocentric-specific approaches, while also highlighting the value of separately evaluating static and dynamic regions of a video.

Visual counting is an important task in computer vision with broad applications in areas such as crowd monitoring, agriculture, and environmental analysis. While deep learning has significantly advanced this field by enabling models to learn robust feature representations, deep learning approaches suffer from sensitivity to data imbalances, which occur in the distribution of object counts across counting datasets as a result of annotation effort. Most state-of-the-art counting models, categorized into clustering-, detection-, regression-, and density estimation-based methods, are built upon Convolutional Neural Networks (CNNs) and Transformers, both of which are known to be susceptible to imbalances in the training data. This study introduces a hybrid model that incorporates a programmatically guaranteed counting mechanism using the RASP language and the Tracr compiler, enabling the construction of Transformer-based models that can reliably execute predefined tasks, such as counting. By combining this exact counting mechanism with a trainable embedding module, we present a model that is capable of learning to count various tokens, even under significant data imbalance. We validate our approach on a synthetic, imbalanced dataset and compare its performance, training time, and data efficiency against standard CNN- and Transformer-based models. Results suggest that our method achieves strong generalization across the full spectrum of object counts while requiring less training data, highlighting the potential for this architecture to be further investigated and adapted to be used for robust and efficient visual counting.

Audio-to-motion generation is an important task with applications in virtual avatar creation for XR systems and intelligent robot control in daily life scenarios.
Most current motion generation methods depend on a single encoder-decoder architecture to simultaneously model all body parts, constraining their capacity to capture the diverse and complex motions exhibited by humans.
In this paper, we propose a novel method, SpeechCAT, that employs three separate encoder-decoder modules to individually model the motions of the face, body, and hands. To capture the relationships and synchronization among these body parts, we introduce a cross-attention mechanism to effectively learn their correlations.
SpeechCAT ensures sufficient capacity to model the unique characteristics of each body part while preserving the coherence between them.
Our experimental results demonstrate the superiority of SpeechCAT over baseline methods, highlighting its effectiveness in generating diverse, realistic, and synchronized motions with face, body, and hand parts.

Early detection of leprosy, a neglected tropical disease, is crucial to preventing irreversible nerve damage and disability. Analyzing temperature vari- ations in hands using infrared (IR) cameras offers a potential low-cost alternative to existing medical equipment for early detection of leprosy. This study explores the adaptation of hand landmark detec- tion models, commonly used for hand pose track- ing, to infer the hypothenar area, a critical region for leprosy diagnosis. The research addresses the challenge of limited ground-truth data for the hy- pothenar keypoint by developing annotated datasets and evaluating machine learning models like Lasso Regression and XGBoost. These models signif- icantly outperform the existing method of linear interpolation, demonstrating the feasibility of ac- curate hypothenar keypoint prediction even with limited training data. The findings contribute to the development of accessible, automated tools for early leprosy diagnosis, particularly in resource- constrained settings.

Hand landmark detection in infrared (IR) images is essential for early leprosy diagnosis in developing countries like Nepal, helping to prevent serious complications and disability. However, current hand landmark detection models, such as Google’s detection models comprised in the MediaPipe framework, often struggle with this task due to domain mismatch. While these models are trained on RGB images, the data for this research consists of greyscale IR images. This study addresses this challenge by exploring image transformation and colorization techniques to enhance MediaPipe's hand landmark recognition accuracy on IR images. Preprocessing was chosen over retraining the existing model due to limited computational resources and the lack of labeled target domain data, which makes the retraining infeasible.

Two preprocessing pipelines were developed to address different image characteristics: images with visible hand edges but varying colors of the hand, and images where hands blend in with the background, making the edges difficult to distinguish. The transformations include turning an image into its negative, colorization, contrast enhancement using Contrast Limited Adaptive Histogram Equalization (CLAHE), and masking to remove occlusion.

To evaluate the effectiveness of these techniques, accuracy has been calculated using Percentage of Correct Keypoints (PCK) metric and were compared against two baselines: a lower bound (MediaPipe performance on unchanged IR images) and an upper bound (MediaPipe performance on similar RGB images). Preliminary findings indicate that colorization significantly improves recognition for hands with sharp color transition, while contrast enhancement boosts edge definition for hands that blend into the background. By combining these approaches, the overall accuracy of hand landmark detection improved up to 25%, depending on the threshold value, particularly for the targeted open palm-up hand position.

These results demonstrate that preprocessing techniques can effectively reduce the input domain mismatch, enhancing automated leprosy diagnosis and supporting early detection efforts in low-resource settings.

Leprosy remains a significant health challenge in developing countries, where early diagnosis is crucial to prevent severe disabilities and social stigma. Recent studies have shown that infrared imaging can be used to detect abnormalities associated with leprosy by analyzing hand temperature variations. However, existing diagnostic methods relying on manual annotation of thermal images are timeconsuming, lack standardization, and require technical expertise. This research investigates methods for implementing real-time infrared video-based temperature analysis on mobile devices by focusing on hand landmark detection models, model optimization techniques, and evaluation metrics. A comprehensive literature review identified promising models such as MediaPipe Hands, OpenPose, and YOLO variants for hand landmark detection, along with optimization methods like pruning, quantization, and Neural Architecture Search (NAS) to adapt these models for mobile deployment. Furthermore, evaluation frameworks incorporating both performance and capability-oriented metrics were examined to ensure efficient and reliable deployment on resource-constrained devices. This study provides insights into developing a fully automated, mobile-based diagnostic tool for early leprosy detection, highlighting the challenges and opportunities in adapting visual AI models for infrared analysis. Future research should focus on empirical validation of optimized models on mobile platforms.

This study investigates sensor technologies for di- agnosing leprosy in Nepal, focussing on skin tem- perature in the hands using contact and non-contact sensors. Leprosy affects the peripheral nervous system, causing thermoregulatory dysfunction de- tectable via localized skin temperature changes. A systematised comparative review compares contact thermometry, infrared (IR) thermography, and IR- RGB thermography based on measurement quality, usability, and cost. Next to the systematised re- view, an experimental method is proposed to com- bine RGB and IR imaging to enhance the spatial accuracy of automatic region of interest (ROI) de- tection using MediaPipe Hand Landmarker. The study introduces a multimodal dataset of 45 sets of annotated IR-RGB images and validates a geomet- rical image registration model, achieving 93.2% keypoint detection accuracy—significantly outper- forming IR-only sensors. Results show IR-RGB thermography as a cost-effective, flexible, and ac- curate tool for early leprosy diagnosis in resource- limited settings.