Recent advancements in RGB-only dense Simultaneous Localization and Mapping have predominantly focused on combining a dense scene representation, based on 3D Gaussian Splatting (3DGS), with a camera pose estimation and per-frame depth prediction module. Although these methods have made progress in accurate camera tracking and photorealistic reconstruction quality, they still require large amounts of computational resources, which makes them unsuitable for resource-constrained applications. To this end, we propose dual-scene representation with a novel camera pose optimization module that uses a sparse point-based scene representation, optimized using multi-view point tracks from a pre-trained network. We combine this camera tracker with a 3DGS-based dense scene representation to achieve accurate camera pose estimations and high-quality scene renderings with significantly lower GPU memory usage. We evaluate our method with quantitative and qualitative results on a synthetic and real-world dataset, achieving competitive performance with state-of-the-art GPU memory usage.

The efficient rendering of scenes with many light sources remains one of the most challenging problems in real-time ray tracing. As the complexity of virtual environments continues to increase, with some scenes containing thousands of light sources, traditional Monte Carlo methods struggle to achieve acceptable noise levels under real-time constraints. This paper introduces a novel approach that combines Reservoir-based Spatiotemporal Importance Resampling (ReSTIR) with a specialised bounding volume hierarchy (BVH) structure to enhance light candidate generation for scenes with many light sources. The BVH-assisted candidate generation is tested on multiple scenes, resulting in a significant decrease in image noise levels measured with the root mean squared metric (RMSE), along with improved visual quality from the first frame onward, especially in scenes with numerous light sources illuminating local areas.

In this paper, we propose techniques to increase the effective resolution of a pathtracer without changing the resolution of the screen for which it is rendered. These make use of the fact that displays are made up of pixels, each of which is made up of three distinct subpixels. By setting luminance values for subpixels rather than colour values for a full pixel, we can create a higher fidelity image. This is already used in rendering text, but has not yet been applied to a pathtracer. It is done in three distinct ways; firstly we send three rays per pixel, each centred at a subpixel to obtain a value for every subpixel, the second method is using a filter used for downscaling text. Lastly, we implement a technique that makes use of the aforementioned filter without increasing the number of samples needed. By sampling a random subpixels each frame for each pixel and combining neighbouring pixels proportional to the filter. We find that these methods are quite effective at the goal of increasing perceived resolution in high-contrast scenes.

This thesis addresses the challenge of initial frame noise in real-time ray tracing when using ReSTIR. We propose and evaluate an approach that integrates a normal- aware hash grid for precomputed reservoir caching to improve direct illumination. The research investigates how reservoir caching enhances visual quality alongside ReSTIR and analyzes the associated trade-offs in memory usage and performance. Our con- tribution includes the implementation and analysis of this caching strategy, assessing its impact on visual fidelity across diverse scenes. Although the method significantly reduces noise and improves initial sampling convergence, it can introduce visible grid artifacts in scenes with many light overlaps. Furthermore, this approach incurs notable memory overhead and increased frame times. This work demonstrates the potential of normal-aware hash grids for ReSTIR improvements, providing a proof-of-concept algorithm for stable, high-quality initial samples.

Realistic image synthesis in computer graphics relies heavily on global illumination (GI) to simulate indirect lighting. Computing GI remains one of the most computationally demanding tasks. Traditional approaches like Instant Radiosity (photon mapping) generate virtual point lights (VPLs) to approximate indirect lighting, but become inefficient in complex scenes due to reliance on uniform light sampling. As the number of VPLs grows, this method leads to increased noise and reduced performance.
In this paper we explore the integration of ReSTIR DI (Reservoir-based Spatiotemporal Importance Resampling for Direct Illumination) with photon-mapped VPLs to improve the efficiency and quality of global illumination. By leveraging ReSTIR’s ability to reuse and resample light paths spatially and temporally, the aim is to address the scalability issues of uniform sampling and enable practical, real-time GI rendering.
The approach is evaluated using three test scenes of varying complexity: Cornell Box, Sahur, and a detailed living room scene. Performance metrics include root mean squared error (RMSE) comparison against a reference and average frame times across different sampling methods. Results demonstrate that ReSTIR with photon mapping achieves significantly lower RMSE values and faster convergence compared to uniform sampling, with an average RMSE improvement of 57.4% across all tested scenes. The method shows particularly strong improvements in visually complex scenes, with ReSTIR consistently producing the cleanest results at 1 sample per pixel. However, the integration within the current implementation introduces computational overhead, with frame times increasing by 4-7 times compared to uniform sampling, and occasional blotting artifacts in regions with high VPL density. Despite these limitations, the work establishes that ReSTIR DI can be successfully extended to handle photon-mapped global illumination, providing a foundation for improving VPL-based rendering efficiency while maintaining visual quality.

Spectral Monte Carlo rendering stands as the gold standard for accurately simulating complex optical phenomena, such as fluorescence and chromatic dispersion, but typically exhibits slow convergence behavior due to challenges in sampling the wavelength domain. Convergence is even slower for scenes with complex spectral distributions, and hence a robust method for sampling the wavelength domain is crucial for better performance. To address this challenge, we propose a preprocessing step consisting of optimizing a set of distributions specifically for wavelength sampling, and we investigate whether this approach can achieve a lower perceptual error than traditional sampling techniques. Our evaluation indicates that our method can significantly reduce perceptual error in single-emitter scenes featuring a complex spectral power distribution (SPD), when compared to uniform sampling, without incurring any additional overhead during rendering.

Self-supervised learning (SSL) is a promising approach for medical imaging tasks by reducing the need for labeled data, but most existing SSL methods treat each scan as an isolated sample and overlook the fact that patients often have multiple radiographs taken over time. These longitudinal sequences—multiple scans of the same hip acquired at different visits—encode the natural progression of osteoarthritis (OA) and thus could enrich representation learning. In this study, we evaluate whether incorporating temporal information from these longitudinal radiographic sequences into SSL pretraining yields more transferable representations and leads to improved downstream classification of hip OA severity. We focus on a temporal contrastive task (Contrastive Predictive Coding, CPC), which learns to predict future scan representations from earlier ones, and compare it to a SimCLR-based pretraining that treats each radiograph independently. We also investigate a multitask framework that combines both objectives — either by sequentially pretraining with CPC then SimCLR, or by interleaving the two tasks. Experiments on the Osteoarthritis Initiative (OAI) dataset for binary classification of KL-grade severity show that CPC alone does not surpass SimCLR-based pretraining. However, both the sequential and interleaved multitask approaches significantly improve classification accuracy over either single-task method. These findings demonstrate that even though temporal prediction by itself isn’t sufficient — combining temporal and within-scan contrastive learning can yield stronger models for hip OA severity assessment.

Egocentric video provides a unique view into human perception and interaction, with growing relevance for augmented reality, robotics, and assistive technologies. However, rapid camera motion and complex scene dynamics pose major challenges for 3D reconstruction from this perspective. While 3D Gaussian Splatting (3DGS) has become a state-of-the-art method for efficient, high-quality novel view synthesis, variants, that focus on reconstructing dynamic scenes from monocular video are rarely evaluated on egocentric video. It remains unclear whether existing models generalize to this setting or if egocentric-specific solutions are needed. In this work, we evaluate dynamic monocular 3DGS models on egocentric and exocentric video using paired ego-exo recordings from the EgoExo4D dataset. We find that reconstruction quality is consistently lower in egocentric views. Analysis reveals that the difference in reconstruction quality, measured in peak signal-to-noise ratio, stems from the reconstruction of static, not dynamic, content. Our findings underscore current limitations and motivate the development of egocentric-specific approaches, while also highlighting the value of separately evaluating static and dynamic regions of a video

When optimising the placement of sensors on an Autonomous Vehicle (AV), research often uses evolutionary algorithms, offering a flexible way to explore complex solution spaces with multiple candidate configurations. However, this approach limits the ability to optimise one particular configuration directly. Sensor placement optimisation methods generally aim to optimise sensor-related aspects, such as visibility or coverage. When sensor placement is optimised based on the performance of a downstream task, this task performance is generally implicit, using surrogate metrics such as important task elements or probabilistic-related metrics. To address this, we propose to use gradient-descent-based optimisation in combination with a differentiable renderer. Making rendering differentiable allows gradients to flow between the initial rendering settings and the output rendered image, enabling gradient-based optimisation methods to optimise parameters based on the gradients of the objective function. This combination allows us to more directly optimise the placement of sensors on an associated downstream task. For this approach, we optimise the position of cameras on an AV based on 2D object detection performance. We create triangle mesh representations of traffic scenes from an AV simulator to use with the differentiable renderer. We use an objective function that combines optimisation losses related to in-frame rotational differences and visibility based on the detected object area of non-ego vehicles. This optimisation also considers constraints such as a minimum distance between each camera pair and camera positions that do not extend across the positioning bounds. The objective formulation results in an improved mean Average Precision (mAP) compared to a random sampling strategy and an intuitive baseline represented by cameras around the highest point on the ego vehicle.

The prevalence of partial differential equations (PDEs) in modeling physics and the low speeds of numerical solvers demands more efficient solving methods. For this purpose, machine learning based methods have been proposed, but these are typically discrete, difficult to interpret and require large amounts of ground truth data to train. To address these issues, we propose a novel machine learning based solver that builds upon advancements in Gaussian based reconstruction. Our method represents the solution to a time-dependent target PDE purely in terms of Gaussians, making it completely continuous and meshless. These Gaussians are evolved by means of an autoregressive neural network that is applied to each Gaussian, integrating information from a local neighborhood of Gaussians. This enables the change of position, scale, rotation and value of the Gaussians to model the solution in a particle-like manner. Extensive experiments show the potential for Gaussians to model arbitrary physical phenomena. We also compare our approach to ground truth data and various state of the art methods, which demonstrates that the method performs well on short-term prediction, but does not match the state of the art for long-term prediction.

Multi-view image recognition is crucial for numerous applications such as autonomous vehicles and robotics, where accurate 3D reconstructions from 2D images are essential. However, the presence of various noise factors like motion blur, variable lighting, and changes in the field of view can significantly degrade the quality of these reconstructions. Through a series of experiments using simulated data created in Blender, I explore the effectiveness of different de-noising methods, including AI-based and traditional image processing techniques. My findings indicate that specific adjustments in the Gaussian Splatting data preprocessing can significantly mitigate the adverse effects of noise, leading to more accurate and reliable 3D reconstructions. The study contributes to the field by providing a detailed analysis of noise impact and proposing viable solutions to enhance the fidelity of multi-view image reconstructions in noisy environments. The code and resources developed for this project are available in the GS-preprocessor repository on GitHub. https://github.com/surftijmen/GS-preprocessor.

Convolutional Neural Networks (CNNs) benefit from fine-grained details in high-resolution images, but these images are not always easily available as data collection can be expensive or time-consuming. Transfer learning pre-trains models on data from a related domain before fine-tuning on the main domain, and is a common strategy to deal with limited data. However, transfer learning requires a similar domain with enough available data to exist, and transferability varies from task to task. To deal with limited high-resolution data we propose resolution transfer: using low-resolution data to improve high-resolution accuracy. For resolution transfer, we use Continuous kernel CNNs (CKCNNs) that can adapt their kernel size to changes in resolution and perform well on unseen resolutions. Training CKCNNs on high-resolution images is currently significantly slower than CNNs. We lower the inference costs of CKCNNs to enable training on high-resolution data. We introduce a CKCNN parameterization that constrains the frequencies of kernels to avoid distortions when the kernel size is changed, improving resolution transfer accuracy. We improve fine-tuning with a High-Frequency Adaptation module that complements our constrained kernels. We demonstrate that CKCNNs with kernel resolution adaptation outperform CNNs for resolution transfer tasks with no fine-tuning or with limited fine-tuning data. We compare to transfer learning, and achieve competitive classification accuracy with an ImageNet pre-trained ResNet-18. Our method provides an alternative to transfer learning that uses low-resolution data to improve classification accuracy when high-resolution data is limited.

Neural Radiance Fields (NeRF) and their adaptations are known to be computationally intensive during both the training and the evaluating stages. Despite being the end goal, directly rendering a full-resolution representation of the scene is not necessary and not very practical for scenarios like streamed applications. Our goal is to design a streamable adaptation for a model that can produce fast, rough estimates of 3D scenes, by only using a shallow part of the network. The quality is subsequently improved as more parts of the network are available, such that it can be used in online applications where the model needs to be transferred. Separate models can be trained at different resolutions, but this approach results in a large space overhead and also increases the evaluation time. This can be mitigated by reducing the depth of low-resolution models, but redundancy will still be high as each new model needs to re-evaluate the input data, rendering previous calculations obsolete. Our method combines key concepts from previous approaches to create a progressively trained model that is able to produce intermediate outputs of increasing quality while attempting to optimize the trade-off between overhead and quality. Our model is able to produce a recognizable representation of the scene with as little as one hidden layer from the original model. It also allows for division into streamable chunks which can be sent individually and, upon reconstruction, provide intermediate outputs that bring consistent improvement in quality. The newly streamed data uses the residual output from previous computations in order to reduce redundancy. We show that the final quality of our adaptation is within 2% of the original in terms of previously used quantitative metrics.

Neural radiance fields (NeRF) based solutions for novel view synthesis can achieve state of the art results. Recent work proposes models that take less time to render, need less training data or take up less space. However, few papers explore the use of NeRFs in classic rendering scenarios such as rasterization, which could contribute to wider adoption. Our paper tackles the issue of shadow generation and proposes a deep residual MLP network with fast evaluation times, that generates view-dependent shadow maps. The network distills the knowledge of an existing NeRF model and achieves the speedup through the use of neural light fields, by only doing one network forward per ray.

Radiance fields are a promising alternative to conventional 3D representations in the domain of novel view synthesis, with recent research achieving truly impressive photorealistic view synthesis results. In this paper, we deal with the concept of non-photorealistic rendering in the context of radiance fields, for generating more stylistic captures of real-world objects. We discuss the suitability of inverse rendering as a stepping-stone to traditional shading and edge detection, and contribute a new algorithm specifically for outline detection in radiance fields.

With the current state-of-the-art research, exporting a NeRF to a mesh has the side effect of having to evaluate a Multi Layer Perceptron at render-time, causing a significant decrease in performance. We have found a way to use K-Means clustering to pre-compute values for this MLP, storing them in multiple octahedron maps for the GPU to fetch when it's time to render the object. This improves render times by a factor of 3-4x.

Generating synthetic images has wide applications in several fields such as creating datasets for machine learning or using these images to investigate the behaviour of machine learning models. An essential requirement when generating images is to control aspects such as the entities or objects in the image. Controlling this helps in creating custom datasets tailored for the above applications and creates a platform to conduct diverse experiments with the generated images, enabling research in the application's field.Existing methods individually enable controllability over various elements in the image such as selecting the objects, their properties, colour or the relations between objects, etc. but we identify a research gap in this field where no single method allows the user to control all these aspects of the image. An additional research gap identified is that existing methods cannot generate images based on a query with a logic based combination of entities.

In this thesis, we aim to fill this research gap by developing SceneUI - a system that allows the user to specify and control aspects of the scene through a user interface such as the objects, object properties, spatial relations between objects, object colour, extent of contextual objects and the background of the image. Additionally, we include a component where the scene is generated based on an OR query specifying the objects as predicates, which serves as a foundation to generating images based on entity combinations. Owing to the limited range of attributes for objects and the lack of objects with attributes in the dataset, we augment the dataset by expanding the attributes of objects in the scene graphs of the dataset and introducing additional objects that have attributes. This was done by identifying recurring objects in the dataset that could be expanded and manually annotating the changes in the dataset. This increases the level of controllability and gives a wider range of an object's properties to choose from.

The goal of the thesis is to design and develop a method that allows the user to declare, specify and manipulate elements of the image and eventually use the images generated for two use cases - Generating Images for Interpretable Machine Learning and Generating Images from Queries as Ground Truth. We evaluate SceneUI to show its usability and effectiveness for the two use cases through two experiments. In the first experiment, we use SceneUI to create biased datasets where each bias is based on an object's colour or object type and the goal is to train a deep learning model that learns the biases. The results show that the model learns the biases well and thus, SceneUI can be used to control datasets which can be used to benchmark machine learning explainability methods. The second experiment generates a dataset of images using the OR query and training machine learning models on the dataset to ensure the suitability of the images for machine learning tasks. As the generated images will be used as ground truth given a query for a specialised machine learning model, the model is expected to identify the predicate objects in the image. The results show SceneUI can generate images based on objects in the query and can also accommodate the objects to have properties. The models identify important features in the SceneUI-generated images and are thus suitable to be used as ground truth. We also discuss the limitations and tradeoffs of SceneUI and present potential future directions for research to improve its scalability.

In this work, we look at the intersection of Sustainable Software Engineering and AI engineering known as Green AI. AI computing is rapidly becoming more expensive, calling for a change in design philosophy. We consider both training and inference of neural networks used for image vision; to reveal energy-efficient practices in an exploratory fashion.

First of all, we examine a modern algorithm for hyperparameter optimisation and compare this to two baseline methods. We find that the baseline algorithms perform considerably worse despite their wide usage and argue that they should not be used when training large models. Furthermore, we look at the layer structure of convolutional networks and conclude that the convolutional layers have the largest influence on the total consumption. We report increases of up to 95% with only marginal improvements in accuracy. Therefore we recommend developers to reduce their network architectures as long as the performance stays within a reasonable margin.

Second, we present a study focused on the inference phase of the deep learning pipeline. We look at the effect of batching for image classification requests. To facilitate the data collection, we make use of a simulated queue and the Pytorch framework. We find that batching has a significant impact on the energy consumption, but the magnitude of this impact can vary a lot for different models. Our recommendation is to treat the batch size as an inference parameter that needs to be tuned first. Additionally, we highlight how the energy consumption of image vision networks has evolved over the past decade. Presenting the findings together with the performance of these networks shows a steady, upward energy trend accompanied by a decreasing slope for the accuracy. The only exception is the model ShuffleNetV2. We mention the design principles that went into the development of this network and present it as a start for future research.

The appearance of an object or scene is determined by factors like the material, the lights, the geometry, the position of the observer, and the surroundings.
Changes in these factors can be simulated using a projector-camera setup.
Other research focuses on changing the appearance from the perspective of the projector, or on projector compensation for slightly warped planar surfaces.
This paper aims to simulate changes in the scene's appearance by actively manipulating the lighting using a projector-camera setup.
It works on not only planar surfaces, but also on more complex geometries.
This is achieved by first doing a calibration, and then using this to optimize a projection.
This projection is optimized to minimize the difference between how the scene looks when the projection is projected and the desired scene.
For low-resolution projectors, it can do this in a few seconds to half a minute.
For higher resolutions, the calibration time and file size get quite big.
This can be solved in future work using different calibration methods.