Appearance-based 3D gaze estimation must accommodate two conflicting needs: fine ocular detail and global facial context. Vanilla Vision Transformers (ViTs) struggle with both needs due to their fixed 16 × 16 patch grid that (i) fragments critical features like the eyes into multiple patches, and (ii) floods the self-attention mechanism with redundant information from the forehead, cheeks and background. We introduce FocusViT, a lightweight and end-to-end differentiable framework that enhances ViTs by first using a Patch Translation Module, based on SpatialTransformer Networks, to dynamically translate patches to center on content, and then employing a Perturbed Top-K operator to select only the most informative tokens for processing.

Our experiments show that combining translation and selection reduces the mean angular error (MAE) of a ViTS baseline on ETH-XGaze from 4.98◦ to 4.61◦ while using 75% fewer tokens. Furthermore, by leveraging this token reduction to enable a finer-grained, lossless 8x8 patch grid, we address a key information bottleneck in the ViT-S architecture, achieving a final MAE of 4.42◦. The framework also demonstrates consistent improvements on the MPIIFaceGaze dataset, reducing the baseline error from 5.72◦ to 5.36◦. Extensive ablation studies confirm our central finding: patch translation and token selection are complementary mechanisms that work in synergy to improve model performance.

Spiking Neural Networks (SNNs) have been widely studied as a computational model due to their sparse spiking patterns, asynchronous behavior, and event-oriented processing style. It enables energy-efficient and low-latency processing ideal for real-time tasks, edge computing, and neuromorphic hardware implementations.

However, efficient training of deep SNNs remains a significant challenge, as spikes can easily vanish or lead to an explosion in deep layers if the membrane potentials or the network weights are not properly initialized. The problem of deriving effective initial weights is addressed for deep, feedforward Artificial Neural Networks (ANNs), but still remains underexplored for SNNs. Though the existing ANN-based methods show some benefits for SNNs, the unique computational dynamics of SNNs are often neglected.

Recent studies suggest that the recurrent SNN is a dynamical system that exhibits a critical state, where spikes propagate across multiple time steps, leading to rich information representations, and is believed to be beneficial for neural computation.
Building upon this perspective, this thesis proposes a novel, bio-inspired, criticality-driven initialization method tailored for deep feed-forward SNNs.
We demonstrate the presence of a phase transition in our models, shifting from subcritical to supercritical dynamics.

Moreover, we apply a Hamming distance-based method to assess the network's ability to discriminate between different inputs. The results show that the system effectively maps distinct inputs to corresponding distinct outputs over very deep layer depth at criticality, as indicated by the minimized normalized mean square error (NMSE) between the input and output Hamming distance matrices. Furthermore, we demonstrate that while subcritical and supercritical regimes result in spike vanishing and explosion, respectively, operating at criticality allows spikes to propagate stably throughout the network.

To evaluate the proposed initialization method empirically, we conduct experiments using feedforward SNNs on standard benchmarks such as MNIST and Fashion-MNIST, and further extend the framework to convolutional SNNs for the more complex CIFAR-10 dataset. The results show that, for all datasets, our criticality-based initialization enables higher accuracy and faster convergence in low-latency settings compared to other baseline initialization methods.

Spiking neural networks (SNNs) with Time-to-First-Spike (TTFS) coding promise rapid, sparse, and energy-efficient inference. However, the impact of sample difficulty on TTFS dynamics remains underexplored. We investigate (i) how input hardness influences first-spike timing and (ii) whether training on hard samples expedites inference. By quantifying difficulty via geometric margins and Gaussian-noise perturbations, and modeling leaky integrate-and-fire dynamics as Gaussian random walks, we derive first-hitting-time predictions. We further show that training-time noise, akin to ridge regularization, reduces weight variance and increases expected spike latencies. Empirical results on a synthetic task, MNIST, NMNIST, and CIFAR-10 with spiking MLPs/CNNs confirm that harder inputs slow inference and noise-trained models trade robustness for latency. Our findings align TTFS behavior with drift-diffusion models and provide a framework for balancing speed and robustness in neuromorphic SNNs.

Distinguishing between benign and malignant ovarian cysts is a challenging task that depends on subjective visual markers in ultrasound scans. Current manual methods remain prone to costly misdiagnoses and the application of these methods depend heavily on the clinician's level of expertise. Recent research demonstrates promising applications of Convolutional Neural Networks (CNNs) for ovarian tumor classification; however, we observed that their performance is limited when applied to a diverse and complex dataset. To address this, we propose, implement, and evaluate three improvements to a baseline classifier.

First, we use a deep learning-based approach to remove burned-in medical annotations and introduce a weighted mean squared error (MSE) loss to improve its effectiveness by emphasizing relevant regions. This aims to better recover the original image content prior to annotation and remove annotations which can act as confounders. Second, we enhance classification by fusing image features with two readily available clinical factors at an intermediate stage of the network. Third, and central to this study, we incorporate a segmentation path that acts as a regularizer, encouraging the shared encoder to learn lesion-specific features that benefit the classification head.

These three contributions are informed by domain-specific knowledge of ovarian lesions and collectively demonstrate promising directions for improving deep learning-based models in this setting.

Voltage imaging using genetically encoded voltage indicators (GEVIs) enables high-speed, population-scale monitoring of neural activity, but it suffers from significant noise due to low photon yield and high frame rates. Effective denoising is essential to recover meaningful signals from such data. In this study, we present a comparative evaluation of three state-of-the-art denoising methods, SUPPORT, DeepCAD-RT, and PMD, on both synthetic and real voltage imaging datasets. Our analysis considers spatial and temporal signal quality, as well as computational efficiency. We find that each method has distinct advantages, and the most suitable choice depends on the specific requirements of the imaging application. SUPPORT is well-suited for tasks requiring spatial detail, PMD offers strong temporal stability and speed, and DeepCAD-RT provides an efficient, balanced alternative. These insights aim to support researchers in selecting and refining denoising tools for real-world neuroscience workflows.

Voltage imaging is an emerging microscopy technique that can make neuroscientific research very prominent. The images obtained with this imaging method exhibit a substantial amount of noise.
Currently, the new methods are developed and tested to computationally denoise voltage imaging videos with high efficiency and preservation of the video structure.
This research attempted to investigate how well traditional denoising algorithms, such as different types of blur or diffusion, perform in denoising such videos. Specifically, five well-established algorithms commonly used in biomedical imaging were evaluated: Gaussian filter, bilateral filter, anisotropic diffusion, wavelet filter, and total variation minimization.
The methods were applied to both real brain recordings (HPC2 dataset) and synthetic videos (Broad DSP CellMincer).
Performance was assessed by measuring the structural similarity index (SSIM) and signal-to-noise ratio (SNR).
Results suggest a trade-off between noise removal and structural preservation, with total variation minimization and anisotropic filtering performing particularly well in terms of noise suppression. These classical methods remain relevant for data exploration and visualization.
For the methods to be used in a medical context, a more in-depth research should be carried out on medical data. The deep learning methods remain relevant for the high-precision applications. Code developed for the research is available online at https://github.com/Rpplctns/denoising-voltage-imaging-videos.git.

Voltage imaging enables high resolution recordings of neuronal activity but suffers from low signal-to-noise ratios (SNR), primarily due to photon shot noise. Traditional denoising methods like VST-GAT and Penalized Matrix Decomposition (PMD) offer effective noise reduction but often trade off temporal and spatial resolution. Recently, deep learning-based denoising methods, such as CellMincer, have emerged as promising alternatives due to their ability to learn complex signal models without requiring clean training data. This paper compares the performance of traditional and deep learning methods for denoising voltage imaging data using both synthetic and in vivo datasets. Metrics such as SNR, PSNR, and tSNR were used to evaluate performance. The results show that CellMincer outperforms traditional methods on synthetic data and performs competitively on real in vivo recordings, suggesting the viability of self-supervised deep learning for voltage imaging denoising. PMD remains a strong baseline with robust performance across datasets. This comparative study highlights both the potential and current limitations of deep learning approaches and suggests directions for future improvement.

Voltage imaging is a powerful technique for observing fast neural activity, but it often produces images with a high level of noise, making analysis difficult. Deep learning methods have shown promise in denoising such data, but most require large datasets containing both clean and noisy image pairs, which are hard to obtain in real-world settings. To this end, several self-supervised approaches that rely solely on noisy images have been proposed in the literature. In this paper, three self-supervised denoising models, Noise2Void, AP-BSN, and DeepVID v2, are evaluated on both synthetic and real voltage imaging datasets. For the synthetic data, the performance is assessed using PSNR and SSIM, while for the real data, the temporal signal-to-noise ratio (tSNR), a metric well-suited to voltage imaging, is used. Results show that the self-supervised models are effective at denoising both synthetic and real image datasets. In particular, models which use the temporal information of the videos, such as DeepVID v2, obtain the best results.

In recent years the emergence of Spiking Neural Net- works (SNNs) has shown that these networks are a promis- ing alternative to traditional Artificial Neural Networks (ANNs) due to their low-power computing capabilities and noise robustness. Nevertheless, in recent approaches, they have either strayed away from spike time backpropaga- tion, used discrete time, single spike neurons, and/or limited themselves to shallow networks. These approaches limit the potential of SNNs by sacrificing significant aspects of what makes them special in order to have a functional network. It is for this reason that we believe that, the implemen- tation of residual connections, allowing a model that has multi-spiking neurons, precise time and spike time back- propagation is the path to allow these networks to truly shine. As it will solve one of this model’s most severe lim- itations which prevent it from being used to build deeper networks, the banishing of spikes at a deeper depth. Hence we aim to remedy this problem by feeding inputs from fur- ther up the network to revitalize the spike counts at deeper layers. In this paper, we explore the implementation of residual connections in precise time multi-spiking neural networks as a way of solving the disappearing spikes problem that inhibits spike time backpropagation. We explore two al- ternatives in the implementation of the residual connection and we analyze how they affect the accuracy of the network as the depth increases in both Multi-Layer Perceptrons and Convolutional Neural Networks. We also developed an ar- chitecture to allow for swapping of the fuse function in or- der to take full advantage of the flexibility that precise time provides us. Results show that the implemented residual connections allow for deeper training, which has the poten- tial to aid in the network’s performance, although in some cases some hurdles remain.

Event cameras are bio-inspired sensors with high dynamic range, high temporal resolution, and low power consumption. These features enable precise motion detection even in challenging lighting conditions and fast-changing scenes, rendering them well-suited for optical flow estimation. However, event camera output is sparse and unstructured, making it challenging to process. Transformer architectures have shown to be effective in capturing long-term temporal dependencies and processing sparse input, hence they might be better suited to processing this output by leveraging the fine time granularity inherent to event camera data.
We introduce E-GMFlow, an approach for event-based optical flow inspired by the recent success in terms of accuracy of transformer-based models for frame-based optical flow. We explore the effect of temporal details on the accuracy of this transformer architecture by changing the number of temporal bins in which events are discretized. We observe that the increase in the number of temporal bins generally causes higher accuracy and comment on the limitations of this study.

Computer vision tasks have shown to benefit greatly from both developments in deep learning networks, and the emergence of event cameras. Deep networks can require a large amount of training data, which is not readily available for event cameras, specifically for optical flow estimation. The need for simulating this data in a realistic, physics-driven manner is therefore crucial. This paper compares the state of the art event camera simulators on different criteria, including event timestamp modeling, performance under low illumination, bandwidth simulation, computation speed and various types of noise simulation. We also summarize the shortcomings of some commonly used optical flow event datasets. For generating high-quality, realistic events, The V2E and DVS-Voltmeter simulators have shown to produce the most accurate data.

Event cameras are novel sensors whose high temporal resolution and bandwidth motivate their use for the optical flow estimation problem. However, the properties of event cameras also introduce a vulnerability to flickering. Flickering hurts the perceptibility of motion by overwhelming event data with unrelated information. The single existing event de-flicker method (EFR) is built for scenarios where the relative position of the camera and the flickering object is constant, which is uncommon in motion-heavy optical flow estimation scenarios. Our contribution is a new de-flickering method that incorporates spatial awareness of nearby pixels. We hypothesize this feature to increase robustness to movement, and thus to better improve optical flow accuracy. Compared to EFR our method falters at filtering intensely flickering surfaces, but better preserves the spatial coherence of edges. However, we observe that both de-flickering methods remove much geometric information, especially given slow motion or weak ambient illumination. Our benchmarking shows that neither our method nor EFR significantly affects optical flow estimation accuracy, despite reducing event counts by 50-65%. Overall, we conclude that the niche benefits of spatial filtering are nullified by the result that filtering hardly affects optical flow estimation.

Optical flow estimation with event cameras encompasses two primary algorithm classes: model-based and learning-based methods. Model-based approaches, do not require any training data while learning-based approaches utilize datasets of events to train neural networks. To effectively apply these algorithms, it's essential to understand their respective strengths and weaknesses.
This study compares model-based and learning-based optical flow estimation methods using event cameras, aiming to provide guidance for real-world applications. We evaluated these methods on the MVSEC and DSEC datasets, focusing on their accuracy and runtime. Our findings indicate that model-based methods excel on the MVSEC dataset, characterized by small motions, while learning-based approaches perform better on the more dynamic DSEC dataset. To investigate potential overfitting of learning-based methods to DSEC, we retrained the IDNet and TMA models on the BlinkFlow dataset. The retrained models demonstrated competitive accuracy, surpassing model-based methods which indicates that learning-based models perform better on datasets like DSEC even when not able to overfit. Finally, our analysis on runtime showed that model-based methods achieve real-time performance on CPUs and learning-based methods require a GPU to run in real-time.

Spiking neural networks (SNNs) aim to utilize mechanisms from biological neurons to bridge the computational and efficiency gaps between the human brain and machine learning systems. The widely used Leaky-Integrate-and-Fire (LIF) neuron model accumulates input spikes into an exponentially decaying membrane potential and generates a spike when this potential exceeds a set threshold. A LIF neuron is characterized by learnable input weights and a manually selected membrane time constant 𝜏, which determines the decay rate of a neuron's membrane potential. Previous work introduced the Parametric LIF (PLIF) neuron model with a learnable 𝜏. However, the published experiments only featured a single 𝜏 per spiking layer. This leaves space for exploration of the effect of having a learnable 𝜏 for each neuron. The importance of 𝜏 is given by the trade-off it inherently introduces, prioritizing the neuron's capability to convey information about spatial or temporal features. This work examines the effect of introducing a learnable 𝜏 per neuron with a new initialization method and a new regularization term which incentivizes low variance in each PLIF layer. The experiments are done using the DVS128 Gesture dataset and compared to a baseline model from the original paper introducing the PLIF neuron model. Results are inconclusive but suggest that introducing 𝜏 per neuron does not have a significant effect on the accuracy of a spiking neural network. Moreover, the evolution of 𝜏 during training exhibits interesting behavior and leads to two new hypotheses.

The promise of Artificial Neural Networks has lead to their immense usage intertwined with concerns over energy consumption. This has led to development of alternatives, such as Spiking Neural Networks (SNNs), which allows their implementation on neuromorphic hardware. In effect, the network must be time discretized. SNNs learn through backpropagation, similarly to ANNs, where two main methods exist: spike-based backpropagation and backpropagation through time. This study investigates and compares the effect of varying timesteps on the convergence rate of BATS, an example of spikebased backpropagation, and of SLAYER - an example of backpropagation through time on the NMNIST and MNIST dataset. The results conclude that BATS withstands growing timestep sizes. Although SLAYER maintains a good performance for small timestep sizes, it seems to self-destruct as they grow.

Spiking Neural Networks (SNN) represent a distinct class of neural network models that incorporate an additional temporal dimension. Neurons within SNN operate according to the Leaky Integrate-and-Fire principle, governed by ordinary differential equations. Inter-layer neuronal communication occurs through spike propagation when membrane potentials reach a specified threshold. This unique mechanism renders conventional Artificial Neural Network (ANN) design principles and learning rules inapplicable to SNN. This study presents a theoretical investigation into the impact of time discretization on SNN performance in real-world tasks. We present a network architecture based on the Error Backpropagation Through Spikes model. This approach allows for the transformation of the original system of differential equations into a system of difference equations across multiple time steps. We evaluate our model's performance on the MNIST dataset and identify several phenomena that influence accuracy. Our analysis primarily focuses on gradient dynamics and spectral properties of the voltage signals. These findings contribute to a deeper understanding of SNN behavior and potential optimization strategies.

Event cameras are cameras that capture events asynchronously based on changes in lighting. They offer multiple benifits, but pose challenges in computer vision due to their asynchronous nature and hard to capture ground truth values to compare against. This paper shows the effects training of a state of the art unsupervised learning algorithm Taming Contrast Maximisation for predicting optical flow on a new dataset BlinkFlow which promises improvements in performance of supervised algorithms. This paper aims to see if these improved performances also happen for unsupervised models. Results of this research were inconclusive for the effectiveness of training unsupervised models, but it was shown that pretrained models on DSEC and MVSEC datasets did not perform well on this new dataset.

The increasing computational costs of training deep learning models have drawn more and more attention towards more power-efficient alternatives such as spiking neural networks (SNNs). SNNs are an artificial neural network that mimics the brain’s way of processing information. These models can be even more power-efficient when run on specialized hardware like digital neuromorphic chips. These chips are designed to handle the unique processing needs of SNNs like sparse and event-driven computations. A lot of the state-of-the-art performance of SNNs in recent research has been achieved through supervised learning models that leverage intricate error backpropagation techniques. These models impose specific constraints on the network and rely on continuous time to facilitate the backpropagation process. However, this fix imposes a new challenge when converting these mechanisms on a neuromorphic chip. Because time is discrete on hardware numerical errors can be introduced as we can not calculate the infinitely precise value of variables depending on time. This work proposes a time discretization technique that allows for fast and stable backpropagation and analyses the effects of it on the efficiency of the SNN. Specifically, we look at sparsity, latency, and accuracy as the main factors that explain the power efficiency of the model. The experiments show that choosing a suitable time step size can improve sparsity while maintaining a high level of accuracy. However, too large values affect the network’s ability to learn.

Spiking neural networks have gained traction as both a tool for neuroscience research and a new frontier in machine learning. A plethora of neuroscience literature exists exploring the realistic simulation of neurons, with complex models re- quiring the formulation and integration of ordinary differential equations. Overcoming this challenge has led to the exploration of various numerical integration techniques with the goal of highly stable and accurate simulations. In contrast, training spiking neural networks is often done with simple leaky integrate-and-fire models and rudimentary integration methods such as the forward-Euler method. In this research we explore how more complex numerical integration methods, borrowed from neuroscience research, affect the training of networks based on current-based leaky integrate- and-fire neurons. We derive equations required for the integration process and suggest the use of spike time interpolation. Furthermore, we pro- vide insights into applying backpropagation on numerically integrated networks and highlight possible pitfalls of the process. We conclude that numerically integrated networks can achieve training accuracies close to their theoretical limits, with good convergence and training time characteristics. Specifically, high order integrations achieve robust and computationally viable training. Additionally, we explore the effects of spike time interpolation on network accuracy and use our findings to pro- vide insights into the role of different integration parameters on the effective training of spiking neural networks.

Insects such as Diptera are capable of highly complex aerial maneuvers and rapid responses to environmental stimuli, making them a subject of interest for studies in flight dynamics and motor control. To accurately quantify these movements, high-speed cameras are employed, capturing footage at 5000 frames per second. This results in a vast amount of data, necessitating the development of automated analysis models. In this work, we present a segmentation model specifically designed for the wings of Diptera species, aiming to track their wingbeats with high precision. The model’s performance was evaluated on species included in the training set as well as on species only included in the testset, allowing us to assess its generalization capabilities. Our results show that the model achieves a 95th percentile error of 7.8 pixels, while operating at a speed 2500 times faster than manual human analysis. This significant improvement in speed highlights the potential for our model to facilitate large-scale, high-speed analysis of insect flight, with implications for both biological research and bio-inspired engineering applications.