In this thesis, we address the problem of learning mesh-specific, impulse-dependent fracture patterns in real time. Our approach is based on regressing a distance field over the mesh surface, encoding the proximity of each vertex to fracture lines, which is subsequently segmented into distinct pieces using graph-based methods such as watershed segmentation. The goal is to achieve real-time performance, which is something the current approach does not achieve for large meshes.

We evaluate different neural architectures, comparing a multilayer perceptron to DeltaConv, a graph convolutional model, and find that the MLP provides superior performance. In addition, we assess multiple segmentation strategies and identify watershed as the most effective, followed by hierarchical segmentation. We also find that the segmentation algorithms do not achieve real-time performance for large meshes.

These results highlight the potential of machine learning-based fracture simulations, but also indicate that distance field segmentation is not capable of real-time performance using our tested algorithms. This suggests that future work should focus on directly learning the labels rather than relying on distance fields as an intermediary representation in real-time scenarios.

Recent advances in generative AI have enabled high-quality video generation from text prompts. However, the majority of existing approaches rely exclusively on prompts, making it difficult for an artist to control the generated scene layout and motion. In this thesis, we propose a novel method for geometry-guided Text to Video generation. Our method takes as input an animated mesh sequence and a text prompt and generates a video following both the text prompt and input geometry. Our pipeline consists of two main stages: Firstly, we use an existing text-driven texture generation method to create an initial rough texture for the geometry. Next, a depth-conditioned T2I model is used to generate video frames following the guidance animation, using the generated texture to enforce temporal consistency across frames. By generating video frames rather than directly using the result of the texture generation, our method supports generating deformations from the guidance geometry and variable lighting and by using the texture for feature alignment, we acheive significantly stronger robustness to occlusions and camera motion than existing controllable video-generation approaches. We begin by identifying the failure modes of existing methods through a set of initial experiments, we then use these findings to propose our method and finally evaluate it through a series of comparisons and ablations.

This paper investigates how standard 3D reconstruction techniques can be adapted to work with orthographic projections of pixel art images. Reconstructing 3D models from 2D images is typically done with real world objects. However, little work has explored this problem in the context of pixel art, which has a lower resolution, and uses stylistic colors and shading, making it a problem that requires different or adapted techniques. We implement three reconstruction algorithms based on some common practices in the field: silhouette-based intersection, spatial carving using color consistency, and gradient-based depth estimation. Results show that silhouette intersection is an effective method for simple models, but fails to capture concave regions. Spatial carving addresses this limitation, although it also fails in some use cases. An adapted depth estimation technique is also used, which most accurately captures these regions, although it does not fill in the model as well. We implement a custom Blender plugin to support user annotations and improve accuracy in some ambiguous areas. We conclude that a hybrid approach, where silhouette intersection is combined with depth estimation, gives the most accurate results, and we suggest future work that can be done on the topic, including better color merging principles, adding custom viewpoint orientations, adding support for multiple object reconstruction and using perspective projections.

In pixel art, texturing is the process of adding detail to an object to make it resemble a real material. While texturing is crucial in creating high-quality pixel art, it is an arduous process that is time consuming for artists. We present an algorithm that automates texturing by arranging elements throughout an input image in a way that implies a feature of the reproduced object, outlines its tridimensional shape, and respects the original shading. We additionally present heuristics on how to determine the color of a feature drawn over existing pixel art, and a method to create vector fields from user brush strokes. Aside from easing artists' work, this algorithm is an important step towards the procedural generation of pixel art. Quality is demonstrated by comparing results to manually authored textures.

Pixel art relies on carefully constructed color ramps to simulate shading and depth within limited palettes. However, editing these ramps remains a tedious and error-prone manual process. This research introduces a semi-automatic tool that supports the detection and modification of color ramps in pixel art. The system builds a graph to model relationships between image colors based on perceptual similarity and spatial adjacency, then extracts and validates color ramps using customizable criteria. Afterwards, if a user edits a color, changes propagate consistently along the defined ramps, preserving their visual structure. The method streamlines palette editing while respecting artistic intent, offering both automation and control.

Pixel art can suffer from perceptual artifacts, such as banding and pillow-shading, which result from poor pixel placement and weaken visual quality. Banding occurs when two adjacent pixel segments of different colors align their endpoints along a shared axis. Pillow-shading is a pronounced form of banding, characterized by concentric strips that perfectly follow the object's outline. Prior methods for fixing artifacts in pixel art rely heavily on auxiliary reference images and do not address banding or pillow-shading. We present a semi-automated method that corrects both artifacts using established artistic guidelines and without relying on references. It supports user annotations to resolve ambiguities when multiple corrections are possible and to help preserve the artistic intent. We tested our software tool, Pixel Fixer, on several pixel artworks affected by banding or pillow-shading. Using an error metric that counts pairs of banded pixel segments, we observed consistent reductions across all inputs.
Related dataset 4TU.ResearchData: https://doi.org/10.4121/11446578-c1f3-474c-9cc4-6978a79b675b

Pixel art is an art style that uses low-resolution raster images with few colors. This style was ubiquitous in video games and computer programs in the late 20th century before high-resolution screens and powerful computers were widely available. In order to remaster these old games, or to use pixel art as an intermediate format to create vector images, conversion to vector formats such as Scalable Vector Graphics (SVG) is desirable. Existing tools have several limitations, notably a lack of support for gradients. We introduce a new method that expands upon an existing pixel art vectorization solution to add support for gradients, while maintaining compatibility with SVG and enabling real time editing for ease of use. Our method works by blurring parts of the image and then using various techniques to improve the quality of the output. We test our solution by creating example images with our tool’s automatic generation capabilities and modifying them with the available controls. We conclude that our method is useful for imitating smooth lighting and real-world gradients, enabling artists to create more expressive vector versions of pixel art.

Effective software testing relies on the quality and correctness of test assertions. Recent Large Language Models (LLMs), such as CodeT5+, have shown significant promise in automating assertion generation tasks; however, their substantial computational resource demands limit their practical deployment in common development environments like laptops or local IDEs. To address this challenge, this work explores knowledge distillation to derive smaller, more efficient student models from a larger, pre-trained CodeT5+ teacher. While knowledge distillation has been successfully applied to general code models, its specific application to creating lightweight, locally-deployable models for test assertion generation remains a recognized research gap. Using a dataset that includes assertion input-output pairs and teacher logits, we systematically investigate the impact of different distillation loss components—soft logits loss and hard target losses—on student performance. Our findings demonstrate the practical viability of this approach: a distilled 220M parameter student model can be nearly 3x faster and consume over 40% less memory than its 770M teacher, while retaining approximately 78% of the original’s assertion generation quality as measured by CodeBLEU. These results offer practical insights and a clear pathway for deploying efficient yet effective assertion-generation models suitable for local developer workflows.

Recovering the appearance and physical parameters of elastic objects from multi-view video is essential for many applications that require simulation of the real world. Past methods for this task have provided accurate results in recovering physical properties; however, their re- liance on Neural Radiance Fields (NeRFs) for novel view synthesis means they trade off visual quality for rendering speed. To address this issue, we present a novel framework for the joint appearance reconstruction and physical parameter estimation of elastic objects relying on 3D Gaussian splatting. Our key insight is that dynamic 3D Gaussian kernels extracted from multi-view video can be used to reconstruct the object’s geometry and supervise elastic parame- ter fitting through a differentiable physics engine. Novel views and object behaviours can then be constructed by forward simulating the extracted mesh and using it to drive the Gaussian kernels. We demon- strate that our method is competitive with the state of the art on phys- ical parameter estimation while being better at reconstructing object appearance. Additionally, our method can simulate novel views and object interactions at near real-time rates that outperform past ap- proaches.

The most established and widely used methods for analysing tree images for tasks such as geometry analysis, segmentation and classification often rely on pixels. In this paper, the applicability of analyzing tree geometry based on a graph representation rather than a pixel-based approach is pursued. To do so, 2D renders of different species of trees are converted to spatial graph structures capturing significant points on the tree skeleton. Two independent Graph Convolutional Network algorithms which learn node (coordinate) features are then applied on the obtained dataset to assess the reliability of graph based analysis. The first experiment explores a GCN for assigning correct species labels to the skeleton graph of the original tree image, demonstrating the association between geometry and tree metadata. The second experiment, an unsupervised representation learning, is conducted by using Graph Autoencoders to obtain an embedding for each skeleton graph which can be used to reconstruct partially the same graph, demonstrating the association between GCE latent representation and geometry. Promising results were found in both cases, reinforcing the reliability of the original proposition to rely on geometry as well as pixels for tree analysis tasks.

The objective of this project is to train a model that transforms a tree with its foliage into only its branch structure. This is achieved by employing machine-learning techniques, specifically Generative Adverserial Networks (GANs). By utilizing the proposed method, a predictive model is built that automatically minimizes its own error function through a comparison of a set of input and ground-truth tree images, which are tree images with and without leaves, respectively. The adoption of GANs has shown promising results, both visually and metrically.

L-Systems allow for the efficient procedeural generation of trees to be used for rendering in video games and simulations. Currently, however, it is difficult to engineer grammars that mimic the behaviours of real life trees in 3 dimensions. To be able to deduce them, the skeleton of a tree can be used to train a model and generate an L-system for a given tree in particular. The aim of this paper is to provide a pipeline to isolate these skeletons from images of a tree, using Neural Radiance Fields (NeRFs) to reconstruct the tree, and using Laplacian Based Contraction to retrieve the underlying skeleton. We find that this approach leads to 3-dimensional topologies that very closely resemble the given tree.

Trees are essential components of both real and digital environments. Therefore, it is important to have 3D models of trees that are of high quality and computationally efficient. One way to achieve this is by compressing a high-quality model using billboard rendering, which involves partitioning the tree into multiple planes to produce a similar result to the original. Our study explores the compression of 3D models using an optimization loop and adapting billboard rendering techniques. We use computer vision primitives to render basic models, which we then optimize by adjusting the texture to resemble the original tree. The models consist of multiple upright planes that are rotated around the central vertical axis of the original tree. We use different optimization functions, such as L1 and L2 losses, to determine the best approach. We can improve the initial models by bounding the billboards and limiting their heights and widths to that of the trees. Additionally, we can use double-sided textures for the billboards to allow more flexibility for optimizing different species of trees. However, optimizing multiple tree types performs differently for each species, leading to improvements that only benefit certain trees in specific scenarios. Using quantitative metrics, we determined which models perform best and how similar they are to the original after training. We found that our compressed models generally resemble the original while having only a fraction of the original size.

Image inpainting is a problem that has been well studied over the last decades. In contrast, for 3D reconstructions such as neural radiance fields (NeRFs), work in this area is still limited. Most existing 3D inpainting methods follow a similar approach: they perform image inpainting on the training images and use the inpainted images for further training of the 3D model. Due to inconsistencies in the different inpaintings of the images, the 3D inpainting often becomes blurry. With the advent of 3D Gaussian Splatting (3DGS), we identify a new opportunity for 3D inpainting. As 3DGS is more explicit in nature than NeRF, we can manipulate the 3D Gaussians directly rather than relying on image inpainting. Based on that key idea, we propose a method that works similar to the PatchMatch image inpainting algorithm. We first construct a nearest-neighbour field (NNF) by searching for nearest-neighbour patches throughout the scene that look similar to the area we want to inpaint. After constructing the NNF we copy the contents of the nearest-neighbour patches to the inpainting region and blend them together to obtain the inpainting result. In our experiments we found that our method performs well in terms of texture synthesis but struggles with structure synthesis, similar to the original PatchMatch algorithm. In cases where only texture synthesis is required to inpaint the area our method is able to provide good results, although in some cases pre-processing of the scene is necessary, as we found that better quality inputs (e.g. the scene itself, the surface mesh underlying the scene, and precise masks) drastically improve the results of our method. Moreover, some parameters of the algorithm are highly scene-dependent and by tailoring them to the scene we can further enhance the performance of the algorithm. Besides introducing a 3D inpainting method that directly manipulates the scene contents, our work offers valuable new insights into 3DGS editing in general.

Neural Radiance Fields (NeRF) and their adaptations are known to be computationally intensive during both the training and the evaluating stages. Despite being the end goal, directly rendering a full-resolution representation of the scene is not necessary and not very practical for scenarios like streamed applications. Our goal is to design a streamable adaptation for a model that can produce fast, rough estimates of 3D scenes, by only using a shallow part of the network. The quality is subsequently improved as more parts of the network are available, such that it can be used in online applications where the model needs to be transferred. Separate models can be trained at different resolutions, but this approach results in a large space overhead and also increases the evaluation time. This can be mitigated by reducing the depth of low-resolution models, but redundancy will still be high as each new model needs to re-evaluate the input data, rendering previous calculations obsolete. Our method combines key concepts from previous approaches to create a progressively trained model that is able to produce intermediate outputs of increasing quality while attempting to optimize the trade-off between overhead and quality. Our model is able to produce a recognizable representation of the scene with as little as one hidden layer from the original model. It also allows for division into streamable chunks which can be sent individually and, upon reconstruction, provide intermediate outputs that bring consistent improvement in quality. The newly streamed data uses the residual output from previous computations in order to reduce redundancy. We show that the final quality of our adaptation is within 2% of the original in terms of previously used quantitative metrics.

Neural radiance fields (NeRF) based solutions for novel view synthesis can achieve state of the art results. Recent work proposes models that take less time to render, need less training data or take up less space. However, few papers explore the use of NeRFs in classic rendering scenarios such as rasterization, which could contribute to wider adoption. Our paper tackles the issue of shadow generation and proposes a deep residual MLP network with fast evaluation times, that generates view-dependent shadow maps. The network distills the knowledge of an existing NeRF model and achieves the speedup through the use of neural light fields, by only doing one network forward per ray.

Radiance fields are a promising alternative to conventional 3D representations in the domain of novel view synthesis, with recent research achieving truly impressive photorealistic view synthesis results. In this paper, we deal with the concept of non-photorealistic rendering in the context of radiance fields, for generating more stylistic captures of real-world objects. We discuss the suitability of inverse rendering as a stepping-stone to traditional shading and edge detection, and contribute a new algorithm specifically for outline detection in radiance fields.

With the current state-of-the-art research, exporting a NeRF to a mesh has the side effect of having to evaluate a Multi Layer Perceptron at render-time, causing a significant decrease in performance. We have found a way to use K-Means clustering to pre-compute values for this MLP, storing them in multiple octahedron maps for the GPU to fetch when it's time to render the object. This improves render times by a factor of 3-4x.

Chromostereoscopic images encode depth as colour, with the red part of the visible spectrum encoding nearby depths and the blue part encoding far-away depths. However, when encoding a regular image with its depth, the generated colours may not match with the original colours in the image. Following a user study, a technique has been developed which takes both the chromostereoscopic effect and the original colours into consideration.

This problem was solved by creating a program to generate chromostereoscopic images from an arbitrary RGBD input image. An optimisation metric was devised to assess if a chromostereoscopic image takes both the target depth and original image into consideration. A user study was conducted to correct this metric with respect to human perception. The new technique can take a desired amount of original colour and chromostereoscopic depth and will generate a chromostereoscopic image, taking these preferences into account.

Chromostereopsis is a visual illusion that can produce perceived 3D images through an effect caused by how humans see different wavelengths. ChromaDepth glasses may be worn to exploit this phenomenon and amplify the effect. Relative to other forms of stereoscopy, chromostereoscopy is lesser-known and has seen fewer applications. This paper seeks to address this, by investigating possible solutions to augment any arbitrary 3D scenes for chromostereopsis. We present techniques to apply the effect in real-time to 3D scenes, while maintaining the scene's original shading, lighting, and optionally some degree of original colour. Additionally, we propose concepts for interfaces that allow user adjustment to applying the effect in areas where user studies have shown variance in preferences. These range from curves that determine varying depth precision, to creating custom chromostereoscopic colour maps, and modifying the speed at which the depth-colour mapping changes, for those sensitive to stereopsis.