Reliable estimation of intra-host viral diversity is essential for understanding viral evolution,
treatment resistance, and outbreak dynamics. However, technical artefacts introduced during
sample preparation and sequencing can distort variant frequencies and lead to incorrect conclusions. One such group of artefacts is ligated chimeric reads, also referred to as ligation chimeras, formed when full-length DNA molecules are erroneously joined during library preparation. Ligation chimeras are currently poorly characterized and their impact on downstream analyses is largely unknown. In this thesis, we developed a modular and reproducible computational pipeline to detect, quantify, and analyze ligated chimeras in amplicon-based viral sequencing datasets. We applied this pipeline to both public and internal datasets, evaluating the prevalence and structural patterns of chimeras and their impact on viral diversity estimates. Our results show that ligated chimeras are widespread, disproportionately affect specific amplicons, and can introduce substantial allele frequency shifts and spurious variants. This means that common filtering strategies in current pipelines risk discarding true low-frequency variants or failing to remove artefactual ones. These findings highlight the importance of chimera-aware preprocessing to ensure accurate viral diversity estimation from long-read sequencing data.

This paper investigates how the local optimization method and strategy affect the efficiency of genetic algorithms (GAs) for Lennard-Jones (LJ) clusters. Several ASE-implemented optimizers were considered; however, only BFGS, FIRE, and Conjugate Gradient (CG) proved viable for integration. The optimizers were first benchmarked independently, and then the GA execution was benchmarked using the different optimizers, with the main solution quality metric being the number of times the global minimum (GM) is found. While BFGS produced better results, its timing was the highest and steepest scaling; furthermore, the trade-off could not be mitigated by reducing the maximal number of optimization steps. In contrast, FIRE and CG, when run with a doubled population size, produced superior results both in terms of execution time and final cluster energy. Furthermore, a cluster isomerism-based heuristic for applying selective local optimization halved the GA's execution time at the expense of a reduction in solution quality. Nonetheless, when computational time was equalized through doubling the population size, the heuristic-based GA produced equal or better results. These findings suggest that faster methods and suboptimal optimization choices, when combined with increased population diversity, can outperform more powerful but slower GA configurations.

Finding the lowest-energy structure of a cluster of atoms is an NP-Hard problem with applications in materials science. Genetic Algorithms (GAs) have shown promise in solving this problem due to their ability to explore complex energy landscapes. A critical component of GAs are the mutations, which maintain diversity and helps avoid premature convergence. Despite the existence of various mutation operations for atomic clusters, there is a lack of benchmarking to compare their effectiveness. In this paper, we evaluate the performance of eight mutation operations within a GA framework for optimizing Lennard-Jones clusters. We assess each mutation based on its ability to reach the global minimum (accuracy) and the time required to do so (runtime). Experiments are conducted across population sizes of 8, 15, and 20, with multiple mutation probabilities tested. Results show that the Etching mutation consistently provides the highest accuracy, but at a considerable runtime cost. In contrast, Twist and Random Displacement offer fast convergence with moderate success rates.

Global Geometry (or Cluster) optimization is the process of finding the most stable formations of a cluster of some atoms. A genetic algorithm was developed to find the global minimum of a cluster using the Lennard-Jones atom interaction model efficiently. Determining the optimal yper-parameters for the algorithm is a computationally intractable task without proper search strategies. Furthermore, during different stages of the algorithms it may be beneficial to change the importance of exploration vs exploitation of the search space. To solve these problems, nine strategies of choosing the algorithm’s mutation rate are presented and benchmarked on the time to find the global optimum. Experimental results show that such strategies can reduce the mean time required to find a global minimum for a given cluster when compared to a constant low mutation rate. However, the results are not statistically significant for all clusters tested. The variance in the results might be caused by the understudied factor of the local optimization the algorithm utilizes. A majority of the algorithm’s runtime is taken up by it, leading to the GA components showing less impact on the results.

Discovery of new materials is essential in a lot if different fields, such as, space exploration, maritime industry and others. To stop new materials undergoing spontaneous reactions or reacting with the environment, they have to stable or at least metastable. That is where Global Geometry Optimization comes in, which tries to find global minimum on its Potential Energy Surface.
But modeling atoms is difficult as there is complex interplay of forces between each atom. A common simplification is to use the Lennard-Jones potential, which treats atoms as 3D points and models their interactions based on distance.
To find global minimum, Genetic Algorithm will be used and this research plans to study multiple different crossover strategies(sphere cut splice, uniform, two point, one point, arithmetical, plane cut splice crossover) and initialization strategies(box, sphere, octahedron initialization) to determine, which is the best for finding minimum for Lennard-Jones clusters. After analyzing results, it is inconclusive, which crossover produces best results, but it is clear that arithmetical produces worst results and octahedron initialization produces best results for clusters with size 10 to 15.

Efficient and inclusive data reuse across research disciplines is based on high quality metadata that bridges the gap between data producers and consumers. This gap, referred to as the metadata gap, arises when the metadata provided by producers do not meet the needs of consumers. Through a comprehensive analysis of metadata supply and demand, this thesis identifies the motivations and barriers faced by producers in creating metadata, along with the challenges consumers face when reusing datasets. To address these issues, the thesis introduces context-bridging data conversations, a framework designed to make metadata creation a more collaborative and adaptive process. The proof-of-concept is built on four key mechanisms: involving consumers as co-creators, recognising and incorporating contextual metadata, leveraging real-time dialogue, and dynamically adapting metadata elicitation questions. Qualitative interviews were conducted to identify the factors that shape metadata practices, and AI-generated summaries were evaluated as a scalable tool to synthesise the insights of these conversations. The findings are applied to the data management plans of CropXR, an interdisciplinary research institute. This case study illustrates how the metadata gap analysis can identify specific areas for improvement in metadata practices and how the context-bridging data conversations framework can provide actionable recommendations to enhance data reusability. By analysing data reusability as a dynamic and context-dependent process, this thesis advances both practical methodologies and theoretical understanding of metadata management. These contributions offer actionable strategies to close the metadata gap, foster collaboration across scientific domains, and promote more efficient and inclusive research practices.

Decision-Focused Learning (DFL) focuses on a setting where a system gets as input some features and needs to predict coefficients to a downstream optimization problem. Classically, one would apply a two-stage solution, which trains the predictor as a regression task and only uses the optimizer during evaluation. However, the two-stage solution fails to optimize the downstream optimization problem. As such, one might use DFL techniques to train the predictor. Nonetheless, these fail to take the entire solution space into account and only optimize toward the optimal true solution, and as such, they might fail to optimize the total downstream value. Furthermore, these techniques are computationally expensive.

We tackle these two problems using Decision Diagrams (DDs) for DFL. DDs for DFL have three main benefits. First, the DD can be cached between runs, speeding up the training loop. Second, we present four novel loss functions that use DDs to reason efficiently over entire solution spaces. Furthermore, we introduce a novel method to relax the DDs to reduce solve-time during training.

We experimentally show that the DDs speed up the training loop substantially. We further show that the DFL losses perform on par with other state-of-the-art DFL losses. Finally, we experimentally show when and which losses work with relaxed DDs.

Although transformers are state-of-the-art models for natural language tasks, obtaining reasonable performance still often requires large transformers which are expensive to train and deploy. Fortunately, there are techniques to increase the size of transformers without extra computing costs. One such technique is sparsity. However, it remains unclear whether sparse architecture is intrinsically more efficient than its dense counterpart. In this paper, we investigate whether replacing the feedforward networks in small transformers with sparse alternatives results in better predictions and faster inference. We found that although inference speed does not increase due to software and hardware limitations, certain sparse alternatives do result in better language understanding. Our research contributes to smarter architectural decision making when designing small language models.

Large language models (LLMs) are rapidly increasing in parameter count, but this growth is not matched by an availability of high-quality data. This discrepancy raises concerns about the sustain- ability of current approaches to language model improvement, especially as forecasts suggest a potential data shortage by the end of the decade. This study investigates the impact of different tokenization strategies on the performance of small transformer (around 10 million parameters) models, evaluating three prominent subword tokenization methods: Byte-Pair Encoding (BPE), Word- Piece, and SentencePiece. Additionally, we ex- amine the trade-off between vocabulary size and embedding size and measure these factors’ effects on language understanding and model efficiency within the BabyLM pipeline with BLiMP and SuperGLUE scores. Our findings indicate that while the different tokenization strategies have minimal impact on model performance, the trade-off be- tween vocabulary size and embedding size significantly affects both language understanding and efficiency. Increasing the vocabulary size beyond a certain threshold does not seem to enhance language understanding. This research improves our understanding of how tokenization influences the language modeling process, specifically within the context of small language models.

Transformers are a type of neural network archi- tecture used in natural language processing. They excel in tasks such as translation, text generation, and language modeling by capturing long-range de- pendencies. Increasing input sequence length en- hances performance but at a high computational cost. This study investigates the effectiveness of Infini-attention, a proposed solution to mitigate these costs, and explores its integration strategies. We implemented and trained Infini-attention on the GPT-NEO platform and TinyStories dataset, evalu- ating on the BabyLM pipeline. Our findings reveal the optimal strategy for integrating Infini-attention.

RNA viruses, characterized by high replication rates and the absence of proofreading mechanisms,
are susceptible to errors during replication. This characteristic allows them to form diverse
communities of genome mutants known as "viral quasispecies". Each individual genome
mutant is referred to as a haplotype. Analyzing viral populations involves reconstructing individual
haplotypes, using sequencing reads. This process is challenging due to the unknown
number of haplotypes, their high similarity, and varied abundances. Common sequencing
methods add to this complexity. Next-generation sequencing provides short reads lacking
sufficient information for haplotype reconstruction, while third-generation sequencing (TGS)
offers longer but error-prone reads. Recent advancements in TGS, such as PacBio HiFi reads,
deliver long, accurate reads, providing new opportunities for haplotype assembly. However,
the majority of TGS viral haplotype assembly tools rely on reference sequences and utilize
alignment-based methods. Moreover, during outbreaks suitable reference genomes might be
unavailable. In the last two decades, machine learning has enabled us to uncover patterns
within and between biological sequences, and to discover important biological attributes. In
this study, we introduce GoViral, a pipeline designed to reconstruct haplotypes specific to
genomic regions and estimate their abundances. Our pipeline features a transformer-based
classification model fine-tuned on a self-constructed dataset to classify read pairs, identifying
those originating from the same haplotype. Predictions are made irrespective of coverage and
abundance, removing reliance on alignment-based methods. In addition, our pipeline employs
a community detection algorithm to cluster and reconstruct region-specific haplotypes, estimating
their abundances. GoViral achieves high performance across real data and diverse
RNA viruses, including SARS-CoV-2, HIV-1, and HCV-1b, surpassing existing tools.

The rapid expansion of large language models (LLMs) driven by the transformer architecture has raised concerns about the lack of high-quality train ing data. This study investigates the role of acti vation functions in smaller-scale language models, specifically those with approximately 10M param eters, to ensure sustained progress in LLM devel opment despite data limitations. Activation func tions, crucial for neural network performance, have evolved significantly, but comprehensive compar isons under consistent conditions remain scarce, especially for smaller parameter count models. This research systematically evaluates traditional and novel activation functions, including learnable variants, and introduces the Kolmogorov-Arnold Network (KAN) to language modeling. Using Hugging Face implementations of GPT-Neo and RoBERTa models, performance impacts were as sessed through the BabyLM evaluation pipeline. The results indicate that activation functions do not significantly impact the performance of these models. Additionally, the model with the KAN network underperformed compared to models with traditional architectures in the context of this study. These findings suggest that optimizing activation functions may not be crucial for smaller language models, emphasizing the need for further research to explore other architectural improvements.

Graph neural networks (GNNs), while effective at various tasks on complex graph-structured data, lack interpretability. Post-hoc explainability techniques developed for these GNNs in order to overcome their inherent uninterpretability have been applied to the additional task of detecting important subnetworks in graphs. For example, the GNN-SubNet program uses explanations of protein-protein interaction networks to detect the most important disease subnetworks for specific types of cancer. However, when using a post-hoc explanation for such additional tasks, evaluating the quality of the explanation becomes critical.
This study implements four explainability evaluation metrics to provide a fast and accurate way of evaluating explainability, using the GNN-SubNet program as a case study of explainable GNNs for subnetwork detection. Fidelity and sparsity metrics are implemented as defined in existing literature, while validity+ and validity- are newly defined. The results show that GNN-SubNet finds robust and faithful but highly dense explanations.

Predicting properties, such as toxicity or water solubility of unknown molecules with Graph Neural Networks has applications in drug research. Because of the ethical concerns associated with using artificial intelligence techniques in the medical field, explainable artificial intelligence techniques are used to explain how GNNs make their decisions. To evaluate the performance of those techniques, different metrics are used. The BAGEL benchmark proposes four such metrics, designed to be useable with any GNN explainer. Of these, the applicability of faithfulness was investigated in molecular property prediction, measured by the submetrics of comprehensiveness and sufficiency. While comprehensiveness and sufficiency were designed to be task agnostic, several shortcomings were identified that make it unsuitable for molecular property prediction. Future recommendations are to investigate other pre-established faithfulness metrics or to develop ones that do not require splitting molecules.

AI explainers are tools capable of approximating how a neural network arrived at a given predic- tion by providing parts of the input data most rel- evant for the model’s choice. These tools have become a major point of research due to a need for human-verifiable predictions in multiple differ- ent fields, such as biomedical engineering. Graph Neural Networks (GNNs) are often used for such tasks, which led to the development of GNNSub- net, a tool capable of finding disease subnetworks on models trained with protein-protein interaction (PPI) data. This tool has been tested with only a single GNN architecture, which left a knowledge gap about the performance of the tool under dif- ferent models, which can differ significantly in the way they operate. Here the question ”How does the explainer perfor- mance vary with change in architectures of training models?” is answered. This paper explores this knowledge gap by train- ing and evaluating two other models (GCN and GraphSAGE) to see if the explanation performance of GNNSubnet changes. The performance is eval- uated with BAGEL metrics, a tool developed for XAI analysis. These metrics allow for comparison of explanations on multiple benchmarks. Three of these - Fidelity, Validity- and Validity+ - measure how accurate an explanation was in terms of iden- tifiying important nodes. The last one - Sparsity - assesses the nontriviality of an explanation by mea- suring how few nodes have been identified as im- portant. The experimental process shows low performance changes with different GNN architectures for accuracy-related metrics - RDT-Fidelity, Validity- and Validity+, which means that GNNSubnet is highly generalizable and not tied to a specific GNN model. However, the Sparsity score differs across models, with GIN being able to provide the most concise - and therefore useful - explanations.

The advent of transformer architectures revolutionized natural language processing, particularly with the popularity of decoder-only transformers for text generation tasks like GPT models. However, the autoregressive nature of these models challenges their inference speed, crucial for real- time applications and resource-constrained environments. Memory bandwidth is a significant bottleneck, especially in autoregressive decoding, where constant loading of large key and value tensors dominates. The Multi-Query Attention (MQA) architecture was proposed to reduce memory access by shrinking the key-value cache, enhancing infer- ence speed at the cost of generation quality. Grouped-Query Attention (GQA) was introduced to mitigate this quality decline, serving as an interpolation between Multi- Head Attention (MHA) and MQA. We explore the trade- offs between inference speed and quality in decoder-only models by experimenting with various proportions of query groups relative to attention heads during pre-training. Additionally, we investigate the impact of reducing the size of key and value vectors compared to GQA and explore a hybrid method combining GQA with shortened key-value vectors. This study aims to expand the list of possible trade-offs and help select an optimal architecture based on specific needs.

The evaluation metrics commonly used for machine learning models often fail to adequately reveal the inner workings of the models, which is particularly necessarily in critical fields like healthcare. Explainable AI techniques, such as counterfactual explanations, offer a way to uncover a model’s internal process. However, these explanations are in literature often used for recourse actions rather than for testing a model’s internal mechanism. In this paper, we propose a proof of concept for a framework which uses counterfactual explanation to evaluate the inner workings of biological machine learning models that use gene expression data. Our approach involves comparing the change of gene expression observed in the original data to the change of gene expression observed between the factual and counterfactual data. The change of gene expression is quantified using the log fold change. Additionally, we expand the definition of faithfulness and introduce a new metric that measures how faithful the generated counterfactual explanations represent the model. This metric should ensure that the explanations accurately reflect the model’s true internal process.

Accurate segmentation of anatomical structures and abnormalities in medical images is crucial, but manual segmentation is time-consuming and automated approaches lack clinical accuracy. In recent years, active learning approaches that aim to combine automatic segmentation with manual input have gained attention in the field, aiming to reduce the annotation effort required for training segmentation models. Batch query retrieval is a key component of active learning as it is a technique that allows for the simultaneous selection of multiple regions/points for annotation. This study investigates the effectiveness of discrete batch query retrieval methods compared to the traditional approach using gradient descent in the context of 3D medical image segmentation. Our experiments show that the active learning paradigm with batch query retrieval provides a few advantages over the gradient descent approach. Additionally, we analyze the impact of discretizing the query retrieval strategy on system performance. Our findings suggest that discretization can lead to slight performance degradation in terms of segmentation quality but offer computational advantages and faster convergence. We also discuss open issues, such as the interpretability of active learning methods, and recommend further research on combining active learning with other segmentation techniques. Overall, our study contributes to the understanding of active learning in medical image segmentation and provides insights for developing more efficient and accurate interactive segmentation models.

Segmentation of 3D medical images is useful for various medical tasks. However, fully automated segmentation lacks the accuracy required for medical purposes while manual segmentation is too time-consuming. Therefore, an active learning method can be used to generate an accurate segmentation using less user input. The active learning pipeline consists of automatic 3D segmentation, generation of a 3D uncertainty field and an optimization algorithm finding the optimal plane in the uncertainty field. This plane can then be shown to a user to be labeled so that the segmentation can be improved. This process is repeated until the user is satisfied with the output. If the plane is chosen so that it contains more errors, then less user input will be needed. This paper focusses on evaluating different optimization algorithms in the context of this pipeline. The three algorithms that are evaluated are particle swarm optimization, gradient descent and L-BFGS-B. The results of the evaluation show that particle swarm optimization converges the quickest, but to lower values than gradient descent. Gradient descent converges slowly, but to high values. L-BFGS-B converges quickly to values that are as high as those from gradient descent. Therefore, using L-BFGS-B in the pipeline instead of gradient descent will decrease the runtime of the pipeline. Using particle swarm optimization will decrease the runtime even further, but at the cost of requiring more user input to obtain a segmentation of acceptable quality.

Although automated segmentation of 3D medical images produce near-ideal results, they encounter limitations and occasional errors, necessitating manual intervention for error correction. Recent studies introduce an active learning pipeline as an efficient solution for this, requiring user corrections only on some of the most uncertain parts of
the automatically segmented image. It does so by combining different uncertainty fields, which are various ways to quantify possible errors. We investigate into its individual uncertainty fields and their combination scheme in attempt to validate its methods. Additionally, we replace its methods for estimating uncertainty with another common way to do so, called the ensemble method, to test possible improvements at uncertainty estimation. Results of this research validates the combination method of the active learning pipeline, and shows weak advantages but strong disadvantages of the ensemble method when compared to the combined method of the active learning pipeline.