Motivation. PAM50 molecular subtype guides breast-cancer treatment but requires sequencing or multi-gene assays that are costly, slow, and often unavailable. Routine H&E histology, by contrast, is universal, and because subtype-driving alterations leave morphological traces, the image itself may carry usable signal. We ask whether H&E whole-slide images can predict molecular subtype as a low-cost triage that prioritizes which patients to send for confirmatory molecular testing, provided each prediction carries interpretable visual evidence rather than a black-box score.

Approach. We present Augur, which predicts PAM50 subtype from a whole-slide image and localizes the tissue behind each call. Because slides are gigapixel with only slide-level labels, Augur adopts the standard weakly-supervised decomposition into two stages: a tile encoder first learns histology features through multi-task pretraining, then an attention-based multiple-instance-learning (MIL) aggregator pools tiles into a slide-level prediction. Our aggregator, DualCLAM, builds on CLAM, an existing model that scores tiles with per-class attention to form a slide representation and adds an instance-clustering loss that refines tile features from slide-level labels alone. DualCLAM augments it with auxiliary mutational-signature regression heads that contribute a molecular training signal and per-signature spatial explanations.

Results. On the TCGA-BRCA cohort under 5-fold stratified cross-validation, the encoder's pretext design dominates accuracy: it adds 0.077 macro AUROC over a segmentation-only baseline, roughly 5 times the effect of any MIL-side change. The best configuration, the four-pretext encoder paired with a CNV signature-regression head, reaches macro AUROC 0.7038. Never trained on annotations, the predicted-subtype attention maps align with pathologist-delineated tumor outlines on held-out slides. The auxiliary signature regression itself, however, is weak: the COSMIC exposure vectors are sparse, high-dimensional, and unevenly covered, ill-suited to the regression objective we used and likely better cast as multilabel classification. Even so, its attention heads still surface a small, stable subset of signatures (CN17, ID14, SBS33, SBS39).

Conclusions. H&E whole-slide images can yield interpretable, spatially-grounded subtype predictions at a level useful for triage. Accuracy hinges on the tile encoder's representation, so encoder pretraining warrants the most effort, but the aggregator should be redesigned, not abandoned: its weak signature branch points to two concrete fixes, predicting only a curated subset of signatures to shrink the decoder and recasting its objective from regression to multilabel classification. By pairing each prediction with visual and molecular evidence, Augur offers a low-cost, auditable pre-screen that prioritizes confirmatory molecular testing. ... Read more

Cell-free DNA (cfDNA) fragmentomics using liquid biopsy data has emerged as a promising minimally invasive approach for cancer detection. While multiple fragmentomic features, including the Fragment Short Long Ratio (FSLR), Motif Diversity Score (MDS), and Copy Number Alterations (CNA), have individually demonstrated predictive value, their complementarity and optimal integration remain insufficiently explored.

In this study, we investigate the relationships between these fragmentomic features in a genome-wide setting and evaluate their complementarity through multi-view intermediate integration for a binary classification task. Variance decomposition with correlation analysis showed that FSLR, MDS, and CNA capture partially non-redundant aspects of tumour-derived cfDNA signals, with only 7.2% overlap among outliers.

This biological complementarity did not translate into substantially improved predictive performance. The strongest downstream model was the PCA-based concatenation baseline using all three views, achieving an AUC of 0.969, with only marginal gains over CNA alone. In contrast, MOFA+ did not improve classification performance, reflecting a mismatch between its variance-maximization objective and cancer–healthy discrimination in cfDNA data. Similarly, Contrastive Multi-View Kernel Learning (CMK) failed to yield separable representations under an unsupervised objective, with meaningful class structure emerging only when supervision was introduced, yet still not surpassing the concatenation baseline.

Across all methods, CNA was consistently the most discriminative single feature, while FSLR provided an additional independent signal. MDS performed worst in all settings and contributed little to predictive performance. This limited contribution may reflect the chosen 5 Mb resolution rather than an inherent lack of biological signal, suggesting that feature-specific resolution optimization should be considered prior to integration. ... Read more

Predictive computational oncology models are fundamentally limited by their uni-modal input drug representations. To overcome this bottleneck, we developed DrugZip, a uniform, task-agnostic, 128-dimensional representation that compresses 25 diverse modalities from the Chemical Checker across a context of 1.2 million molecules. By using a modified autoencoder, DrugZip successfully stabilises the latent space and avoids posterior collapse from a standard variational autoencoder. We evaluated DrugZip across three downstream tasks. In drug synergy prediction, it achieved an AUC of 0.844, resisting performance collapse in unseen cell environments with a mean AUC of 0.62. In drug sensitivity prediction, DrugZip bypassed the extreme overfitting of high-dimensional baselines on unseen drugs. Finally, in cellular perturbation modelling via ChemCPA, DrugZip demonstrated representational sufficiency by matching state-of-the-art transcriptomic prediction accuracy ($R^2$ of 0.776 vs 0.792). Geometrical and information-content analyses confirm that DrugZip produces a continuous, balanced embedding space where drugs remain individually distinguishable. Ultimately, DrugZip shifts the paradigm from engineering task-specific features toward utilising a robust, generalizable, multi-modal representation for computational oncology. ... Read more

Large datasets can today be created with single-cell RNA sequencing (scRNA-seq), allowing researchers to measure RNA expression per cell. In 2026, a new model, Latent Interaction Variational Inference (LIVI), was proposed to analyze these data. LIVI is novel in capturing both cell- and donor-specific variation in the latent space with a Variational Autoencoder (VAE). The research by Vagiaki et al. was primarily focused on discovering expression quantitative trait loci (eQTLs). It is interesting to see how well the cell and donor latent space captures other characteristics, such as treatment success/failure. This research investigates whether the latent spaces of the LIVI model capture major cell types, sub-cell types, and treatment response. A simple classifier (MLP/SVM/Random Forest) was added on top of the latent spaces to evaluate this. Major cell types are clearly distinguishable from the cell latent space C, sub-cell types with a relatively larger class size can be well distinguished in the cell latent space C, and treatment response is partially captured in the DxC space, but not fully separable by a simple classifier. SVM and MLP outperform Random Forest in classifying treatment response. These findings indicate that biologically and clinically relevant information is preserved within the LIVI latent representations. ... Read more

Single-cell expression quantitative trait loci (eQTL) studies link genetic variants to
changes in gene expression in that cell. This allows us to study the effect of genetics on diseases per cell instead of aggregated, since effects can differ per cell type. Traditional SNP to gene expression linking on the single-cell level suffers from the multiple testing burden, due to the great amount of SNPs and genes. To address this, a deep learning framework was developed recently to compress gene expression into low-dimensional encodings and reconstruct the gene expression linearly from these encodings, enabling direct interpretation of the latent space. This model is called Latent Interaction Variational Inference (LIVI). Here, we determine whether the latent factors of this model can serve as a quantitative trait for Single Nucleotide Polymorphisms (SNPs) that associate with Rheumatoid Arthritis (RA) on a dataset with RA patients. RA is a chronic disease characterized by progressive damage of the joints. In this study, we found 617 out of 700 latent factors correlating to at least one SNP, using a linear mixed model. We also found that genes that are associated with RA in a Genome Wide Association Study have a higher loading for associated SNP-Latent factor pairs then for none associated one. We also identified genes affected by GWAS-identified risk SNPs for which the original GWAS did not identify a functionally associated gene. We conclude that the latent factors of the LIVI model can be used as a quantitative trait for SNPs, and used these latent factors to discover trans-eQTLs. ... Read more

Rheumatoid arthritis (RA) is a heterogeneous autoimmune disease: patients who share the same diagnosis respond differently to the same therapy. Zhang et al. stratified the RA synovium into six cell-type abundance phenotypes (CTAPs) by clustering counted, pre-annotated cell-type abundances. The LIVI model, built on a variational autoencoder developed to map trans-eQTLs in non-diagnosed donors, instead learns donor structure directly from gene expression, separating donor-level variation from cell-state variation. The developers of the model left two questions open: whether LIVI can also capture disease status in a diagnosed cohort, and what the optimal number of donor-level embeddings is for a given dataset. We address these by applying LIVI to a CITE-seq dataset of 314,011 cells from 70 RA and 9 OA donors across four different numbers of donor embeddings. In this research, we show that although LIVI is given no cell-type or diagnostic labels, its donor space recovers the underlying cell-type relationships between the six cell types defining CTAPs: a lymphoid (T, B, NK) versus non-lymphoid (myeloid, endothelial, fibroblast) block, which is consistent across all four dimensions. The CTAPs themselves do not form discretely separable groups in the donor space, but at lower dimensionalities, individual donor factors begin to distinguish them along the same axis. Reading the genes behind these factors was limited. We hypothesize this is due to LIVI's sparsity penalty, which was tuned for detecting trans-eQTLs on a much larger cohort, leaving ribosomal pathways dominating the loadings. Therefore, LIVI's donor space captures disease state information, but on a broader scale compared to Zhang et al.'s discretely defined CTAPs. For this particular dataset, the signal becomes stronger at lower dimensionalities, but the interpretability of the signal is limited. ... Read more

Alzheimer's disease (AD) is a complex neurodegenerative disorder and the most common form of dementia, accounting for an estimated 60-70% of all cases worldwide. Understanding how gene regulation changes in specific brain cell types during disease progression is essential to uncovering the molecular mechanisms of AD. Recent research has introduced new methods to extend Gene Regulatory Network (GRN) inference with ATAC-seq in addition to RNA-seq, including ScReNI.

This paper investigates whether integrating chromatin accessibility into GRN inference can help explain gene regulatory changes in specific brain cell types during AD progression by evaluating whether ScReNI's ATAC-derived terms improve biological support beyond RNA-only inference. Using a Python reimplementation, we decompose the published ScReNI formula and assess formula variants on the mouse retina development benchmark with ChIP-Atlas precision and network-clustering ARI. We then apply the same component analysis to SEA-AD MTG data and evaluate whether literature reported AD-related microglia genes survive feature selection.

The results suggest that removing the regulator-locus term and using TF-specific target-peak attribution improves performance of the ScReNI weight formula on the mouse retina development dataset. However, before ScReNI can support strong AD-specific regulatory claims in SEA-AD, feature selection must retain disease-relevant regulators and validation must be performed using human brain cell-type-specific benchmarks. ... Read more

Single-cell RNA sequencing enables the study of biological processes at high resolution, but the high dimensionality and sparsity of its measurements make downstream analyses, such as expression quantitative trait locus (eQTL) mapping, a difficult task. The Latent Interaction Variational Inference (LIVI) model addresses this challenge by learning low-dimensional interpretable embeddings for the cell-state, donor, and donor-cell-state interaction that can be used as phenotypes for association testing. However, LIVI models only gene-expression measurements and does not exploit information from other modalities, such as surface-protein counts that are included in widely used data collection methods such as CITE-seq. In this work, we investigate how LIVI can be extended to jointly model paired RNA and protein data and whether such an extension improves the biological interpretability of its latent representations. We introduce two architectures. Multimodal Shared-space Latent Interaction Variational Inference (MultiSLIVI) is a conservative extension in which RNA and protein measurements share the original cell-state latent space while being reconstructed through modality-specific decoders. Disentangled Multimodal Latent Interaction Variational Inference (DMLIVI) instead separates the cell-state representation into shared and modality-specific components, incorporating disentanglement principles from multimodal variational autoencoders. The models are evaluated using reconstruction performance, cell-type and donor predictability, latent-space structure, and downstream analysis. Most notably, both MultiSLIVI and DMLIVI recover fewer SNP-factor associations than the original LIVI model, indicating that the current multimodal extensions do not improve the donor-factor phenotypes used for eQTL mapping. Nevertheless, the proposed models provide a first step toward multimodal extensions of LIVI and highlight the importance of separating shared and modality-specific variation in future model designs. ... Read more

Rheumatoid arthritis (RA) is a highly heritable disease, yet how its genetic risk translates into cell-type-specific mechanisms remains poorly understood. LIVI is a model that decomposes single-cell expression into donor and cell-state latent spaces, allowing for the reconstruction of the original data, but additionally leaving room for analysis of the retained latent information. The model has been shown to recover polygenic risk signals in healthy cohorts, but whether that is transferable to cohorts with active disease has not been tested. In this work, we apply LIVI to a predominantly RA cohort, with osteoarthritis (OA) patients as control, and ask whether the latent factors carry information about polygenic risk. After first confirming that the clinical cohort cell-state space recovers known immune cell populations, we test each of the 700 donor factors against the polygenic risk scores (PRSs) for 21 diseases, under different testing conditions, and find that one significant factor (D462) is recovered between the latent space and RA PRS. This association survives ancestry correction, and changes in cohort. The factor localises to NK and T cells and drives antigen presentation program whose expression seems to be inversely related to RA risk. ... Read more

ScReNI [21] is a recently developed algorithm that aims to infer gene regulatory networks (GRNs) of single cells based on both single-cell RNA sequencing (scRNA-seq) and single-cell ATAC sequencing (scATAC-seq) data. Because of its novelty, not much is known about its internal mechanisms, which we aim to investigate in this paper.

Specifically, this work compares the highly variable gene (HVG) selection strategy used in ScReNI with a newly proposed approach: type-specific HVG selection. Instead of selecting the top HVGs globally from the entire dataset, we propose selecting the top HVGs within each cell type and inferring the GRN of each cell using only genes specific to its cell type. The comparison is conducted across multiple structural and biological metrics, and the type-specific selection approach shows overall improved performance compared to the original method. ... Read more

Alzheimer’s disease (AD) does not affect all brain cells in the same way, and a
cell’s dysfunction is thought to involve changes in how its genes are regulated rather
than only changes in their expression. Gene regulatory networks (GRNs) model these
regulatory decisions, and recent single-cell methods can infer a separate network for
each individual cell. How such cell-specific networks change in disease, and whether
any changes are shared across cell types or unique to particular ones, remains largely
unexplored. This work asks which regulatory relationships differ between AD and
healthy cells, and to what extent these differences are cell-type specific. To address
this, the cell-specific GRN method ScReNI is reimplemented in Python and applied
to paired single-nucleus RNA and ATAC data from the SEA-AD cohort (27 donors),
inferring one network per cell for microglia and an excitatory-neuron subclass (L2/3
IT). A differential pipeline then compares the networks against disease severity at
the level of individual transcription-factor-to-target edges and of co-regulated gene
modules, complemented by a module-preservation test, treating the donor as the unit of replication. The inferred networks change relatively little with disease: no edge survives stringent correction and weight-based filtering, the module co-regulation structure is preserved, and no module-specific shift is detected in either cell type. The one robust signal is a modest, severity-graded decline in overall regulatory activity in L2/3 IT neurons that persists after adjusting for sequencing depth and is absent in microglia. The results are best read as an absence of strong evidence rather than evidence of absence, motivating larger cohorts and broader gene panels in future work. ... Read more

Prior specification is a fundamental challenge in Bayesian inference. Traditionally, a prior represents a belief over the behavior of a statistical model based on expert knowledge. Such knowledge is not always available, or can be hard to express in a numerical form. As an answer, this thesis introduces the Prior-Learning Prior-Fitted Network (PLPFN), a transformer-based meta-learning model that amortizes prior learning. The model takes in a set of related tasks and outputs fully Bayesian uncertainty estimates over prior parameters in a single forward pass. The novelty lies in providing interpretable prior parameter outputs and examining the prior learning problem through the lens of hierarchical modeling and identifiability. The framework is evaluated on two representative prior structures: Bayesian Linear Regression and a Hierarchical Gaussian Process. The results show that PLPFN's approximations remain close to asymptotically exact baselines, such as closed-form conjugate prior learning and Markov Chain Monte Carlo. Applying the learned priors to Bayesian Optimization shows that the PLPFN is competitive with state-of-the-art prior learning methods in terms of sample efficiency and final regret, both in- and out-of-distribution. The PLPFN framework is a valuable step in shifting prior specification in Bayesian inference from the complex manual translation of beliefs to specifying a set of related datasets. ... Read more

Alzheimer's disease progresses in a spatially heterogeneous manner across the brain, yet how the gene regulatory networks (GRNs) of individual cells vary across space in affected tissue remains largely unstudied. This thesis investigates whether changes in the spatial location of cells in human Alzheimer's disease tissue can be related to changes in GRN structure and the activity of key regulators. Cell-specific GRNs were inferred from paired single-nucleus RNA and ATAC data using pyScReNI, a Python port of the ScReNI algorithm, and projected onto MERFISH spatial coordinates using Tangram, for four cell subtypes across ten overlapping donors from the Seattle Alzheimer's Disease Brain Cell Atlas. First, the spatial mapping was validated as a feasible proof of concept, placing 22-29% of held-out cells within 500μm of their true location. GRN structure was found to be non-randomly organised in tissue space: no signal was present at the level of the whole network, but a subset of leading GRN components was significantly spatially autocorrelated, and the resulting GRN-defined clusters were spatially contiguous beyond chance. This spatial signal was carried by specific regulators concentrated in the neuronal subtypes and absent in the glial subtypes. The most spatially autocorrelated of these, AC106845.1, reached roughly 2.4 times the subtype-median spatial autocorrelation, followed by ADAM28 and FAM189A2, and these regulators operated through an on and off change in activity rather than a rewiring of regulatory targets. A directional but non-significant association with disease severity was observed, which the ten-donor cohort left underpowered. These findings show that single-cell GRNs can be inferred and spatially analysed at a resolution finer than previously available methods, and that their structure is spatially organised by specific regulators in Alzheimer's disease tissue. ... Read more

Large language models evaluated on medical text annotation tasks are typically scored against a single reference label, such as the primary billed ICD-9 code in MIMIC-III. This standard imposes a measurement ceiling on strict accuracy: it conflates genuine clinical extraction failure with cases where the model extracted a valid clinical comorbidity rather than the administratively billed primary code. To address this, we propose a dual-metric evaluation protocol that pairs strict accuracy (matching the primary billed code) with lenient accuracy (matching any billed code for the admission), scoped to clinical ICD-9 chapters, to separate model capability from billing-convention noise. Exercising the protocol through a human-in-the-loop (HITL) framework, we demonstrate an 18 percentage-point gap between strict (0.67) and lenient (0.85) accuracy on a human-reviewed baseline (n = 100); direct on-corpus analysis further shows that approximately 10% of discharge summaries (up to 14% in early iterations) contain gold-label-ambiguous coding artefacts rather than recoverable errors. On the same HITL framework, lenient accuracy converges to a tight [0.86, 0.90] band by iteration 3 across five prompt-engineering conditions, a cross-condition convergence we read as a framework-level reliability gain, and a human-reviewer trajectory shows strict accuracy trending from 0.62 → 0.78 under rich free-text feedback (a directional signal, confounded with batch difficulty at this sample size, that motivates the higher-powered replication we propose). Future LLM-based ICD-coding evaluations on MIMIC-III should adopt the dual-metric protocol so that model capability and billing-convention noise are reported separately. ... Read more

Automated cloud classification is severely bottlenecked by the need for massive, region-specific annotated datasets. This thesis investigates few-shot learning (FSL) approaches for the classification of mesoscale cloud formations across different geographic regions under limited data conditions.

To address this problem, the influence of varying amounts of additional data on generalisation to novel cloud regimes is evaluated. The study compares representation learning strategies, including self-supervised and supervised approaches, to assess their effectiveness in structuring the latent space for distinguishing cloud types. In parallel, two learning paradigms, transfer learning and episodic meta-learning, are analysed to determine how effectively they incorporate additional data when adapting to novel classes.

The results show that self-supervised learning is most effective in the strict few-shot regime, while supervised transfer learning makes the most effective use of additional data. In particular, Barlow Twins achieves the strongest performance under minimal data by avoiding reliance on noisy labels. When additional out-of-domain data, such as ImageNet and auxiliary cloud datasets, is introduced, supervised pre-training combined with transfer learning attains performance comparable to standard supervised learning, while requiring only a small number of labelled examples. ... Read more

Cardiovascular diseases remain the leading cause of death worldwide, yet early detection and continuous monitoring remain challenging outside clinical settings. This dissertation is motivated by the growing potential of remote health monitoring to address this gap—specifically, the use of consumer-grade smartwatches to track cardiovascular health through physiological signals. Although consumer-grade wearables are traditionally merely used as fitness-oriented or recreational, this work investigates the clinical applicability of smartwatch-derived signals for disease monitoring in real-world, non-clinical environments. By enabling scalable, data-driven detection of cardiovascular conditions in everyday settings, such a system has the potential to reduce the burden on physicians, provide patients with continuous insights, and alleviate pressure on healthcare systems through earlier intervention and more personalized care.

By assessing how far wearable-based research has progressed toward operational deployment and identify critical shortcomings in real-world utility and generalizability, we confront several major challenges intrinsic to this domain: the medical interpretability of noisy consumer-grade signals, high inter-subject variability, and the inherent complexity of timeseries data that varies with context (e.g., day/night cycles, physical activity).

Our solution strategy is grounded in machine learning techniques that aim to learn robust, transferable representations of physiological data. In particular, we explore contrastive learning, weak supervision, and morphological modeling—such as acceleration-deceleration curve analysis— as tools to extract clinically relevant patterns. These methods are evaluated across both publicly available and proprietary datasets to ensure applicability to diverse populations.

By addressing these challenges, this dissertation advances the case for smartwatches as viable tools for longitudinal, data-efficient cardiovascular monitoring, contributing to a future in which early detection of conditions like atrial fibrillation and heart failure is feasible at scale in everyday settings.
... Read more

Estimating the effect of an intervention on an outcome is a central challenge across science and society. In medicine, we may ask whether a drug effectively treats a disease, and in economics, whether a new policy reduces unemployment. Estimating such effects from data, a process known as causal inference, is essential but inherently difficult because it often relies on untestable assumptions to ensure unbiased identification of treatment effects. A key example of such an untestable assumption is the absence of unmeasured confounding, meaning that no hidden variable influences both the treatment and the outcome. When this assumption fails, something which we cannot directly verify, treatment effect estimates may become biased. This ultimately can lead to untrustworthy conclusions and, in the worst case, unsafe decisions, such as prescribing the wrong drug to a patient. The central question of this dissertation is therefore whether we can develop methods for safer causal inference that either detect violations of its underlying assumptions or remain robust when those assumptions are violated.
In Part One, we address the first aspect of detecting violations of causal identification assumptions. We focus on settings with data from multiple sources, such as hospitals or locations, where distributional shifts naturally occur. Under specific independence conditions on the causal mechanisms driving these shifts, we first present a nonparametric test to falsify the assumption of no unmeasured confounding. To obtain these results, we introduce a novel technique utilizing hierarchical causal graphical models. Thereafter, we focus on improving the statistical efficiency of this test, which is achieved by reformulating the independence condition using parameterized linear models. Finally, we extend the hierarchical modeling approach to other identification settings, specifically by testing the validity of mediators and instrumental variables used in two additional common identification strategies.
In Parts Two and Three, we develop methods that instead are robust when causal identification assumptions are violated.We revisit two commonly occurring problem settings when doing causal inference and demonstrate that it is possible to develop methods that either remove the need for, or rely on, weaker and more plausible assumptions than those traditionally made. In the first setting, we study the problemof augmenting randomized trials using external data to improve efficiency in treatment effect estimation. Typically, such approaches rely on a transportability assumption that relate the populations underlying the trial and external data. But when this transportability assumption is violated, integrating external data can introduce substantial bias. To address this, we propose a novel and efficient estimator that incorporates external data and show that this estimator improves inference on the average treatment effect while guaranteeing that it never performs worse, and sometimes performs better, than the estimator that relies solely on trial data.We further adapt this estimator to learn heterogeneous treatment effects within the trial population and show that similar safety guarantees hold for this problem.
In the second setting, we examine the evaluation of treatment allocation strategies using Qini curves. Standard methods for estimating Qini curves assume no interference between treated units, meaning that the treatment of one unit does not affect others. However, when interference is present, these Qini curves can be misleading and lead to incorrect evaluation of treatment allocation strategies.We therefore propose multiple estimators to handle the interference, specifically in settings where units within a cluster may affect one another but not units in other clusters.We identify a bias-variance trade-off in these estimators and, through both theoretical and empirical results, provide practical guidance on how practitioners can choose among them. The dissertation concludes with a discussion of broader considerations, limitations of the presented research, and potential directions for future work.We find that it is indeed possible to make causal inference safer by detecting assumption violations and reducing reliance on untestable assumptions. Nonetheless, many open and important questions remain, offering promising avenues for further research on this topic. ... Read more

The field of computer vision research is very large and still growing. Many of these papers concern some type of inductive bias, by proposing new building blocks or alternative training methods for vision models. This type of research has enabled great progress in applications of vision models.

Computer vision concerns itself with the research and development of deep learning models that work on visual data. These vision models are already heavily integrated into society, powering real-world applications such as automated radiology in hospitals, self-driving cars, and autonomous drones. However, it takes a lot of data, in the form of datasets containing thousands or millions of images, to learn reliable vision models. This thesis explores the role that spatial biases (prior knowledge on the position and pose of objects in the image) can play in learning better and more data-efficient vision models.

We find that the practice of integrating prior knowledge on spatial biases (inductive spatial biases) can help to learn biases that are otherwise hard or impossible to learn. Though inductive bias can be difficult and time-consuming to design, and often increases inference cost, integrating inductive bias can result in better performance and greater data efficiency. This work showcases these patterns in spatial biases, specifically position bias and scale bias.

We find that position bias may be learned to some degree by models without the proper inductive bias, but that inductive bias helps to model these biases and improves performance. We show that whether learning position bias is helpful depends on the data. We contribute measures for position bias in vision models in general, as well as in Vision Transformers specifically, to enable the discovery of these findings. We propose an inductive bias on the position embedding of ViTs to better (un)learn position bias.

For scale bias, we find that existing scale-equivariant models for scale bias need to be tuned to the scale distribution of the data. We propose an inductive bias that allows scale-equivariant models to learn the scale bias of the dataset, thereby fitting the data better. We also propose an alternative parameterization of convolutions called MAGNet that can be adapted to known scale distributions present in the data. Models using MAGNets (FlexNets) can be much shallower and do not require pooling.

There are those who advocate against spending much time on inductive biases. The “bitter lesson” of Richard Sutton prescribes that we should simply add more data, not more inductive bias. However, data will run out at some point, perhaps sooner rather than later. Besides raw performance of vision models, given as much data as possible, should not be our only goal: data-deficient settings are real, plentiful, and important. Data-efficient vision models are the future of our field, and the search for appropriate inductive biases will remain an important endeavor. ... Read more

Fermentation has played an important role in the production of foods such as bread, beer, wine, and cheese for thousands of years. These processes rely on micro-organisms that convert sugars into products such as alcohol, lactic acid, and carbon dioxide under oxygen-depleted conditions. Thanks to genetic modification, the variety of substances that can be produced through fermentation has expanded greatly. Fermentative production – also known as bioprocesses – offers a sustainable alternative to fossil fuel-based chemical production. However, metabolic engineering, in which the genetics of micro-organisms are specifically modified to produce desired metabolites in high yields, is highly complex, time-consuming, and costly. One of the main challenges is the enormous design space created by the complexity of cellular metabolism.

Machine learning can help explore this design space more efficiently, for example by predicting the performance of strain designs or suggesting new genetic modifications. In this thesis, we focus on improving these two applications. First, we develop a simulation tool that mimics metabolic processes, allowing us to compare different machine learning models and experimental strategies in a fair and cost-efficient manner. We then use the insights gained from these simulations to optimize yeast strains that produce p-Coumaric acid.

One drawback of many machine learning models is their limited transparency: it is often difficult to understand how a prediction is generated. In this thesis, we investigate how such models can be combined with mechanistic, mathematically formulated models. This hybrid approach brings together the predictive accuracy of machine learning and the interpretability of mechanistic models. We demonstrate that this integration results in more understandable models without sacrificing predictive performance.

Together, the methods and software developed in this thesis provide new tools for applying machine learning more effectively in metabolic engineering, with the aim of accelerating the development of sustainable bioprocesses. ... Read more

Obtaining data is often costly, making it important to assess whether collecting additional data is justified by the expected improvement in performance. Learning curves, which describe the expected performance of a learner as a function of dataset size, provide a useful tool for this purpose. They can help practitioners assess whether further data collection is justified by the anticipated gains. However, additional data does not always lead to improved performance, which makes estimating the potential benefit challenging. Under such conditions, model selection also becomes more difficult, as it is unclear how to compare models when no data is available to evaluate their performance at a hypothetical training set size.

In this context, the thesis takes a step back and asks a more fundamental question: how can we reliably reason about generalization when data is scarce and the behavior of learning curves is itself uncertain? Rather than treating learning curves as simple, and monotonic functions, we study their full statistical structure. We show that variability across training subsets can influence model comparison, decision making, and performance extrapolation. In addition, we investigate conditions under which monotonic improvement can be guaranteed or encouraged. Beyond single task learning, we also examine meta-learning, where information from multiple related tasks is leveraged to improve generalization performance while reducing the amount of data required from any individual task.

We begin by showing that the mean, as a statistical summary of learning curves, may not provide a reliable estimate of performance. We demonstrate that generalization performance distributions are often skewed and heavy tailed, regardless of how they are obtained. As a result, relying solely on the mean for model selection can be suboptimal for some problems.

Next, we propose a semi parametric extrapolation method that adapts its inductive bias to capture complex and potentially non monotonic patterns. This approach improves predictive reliability in settings where additional data collection is costly or infeasible and where learning curves may not exhibit monotonic behavior.

We then study the monotonicity of learning curves under specific conditions. For linear regression, we show that a single gradient update is sufficient to ensure monotonic improvement, provided that the learning rate does not exceed a certain threshold. To construct similarly monotonic learners in practice, we propose a data driven approach for selecting both the learning rate and the initial parameter estimates.

Finally, we investigate the learning curves of a meta learning algorithm. Through controlled synthetic experiments, we analyze the generalization performance of both meta learners and task specific learners, providing insights into how properties of the task distribution influence generalization under a limited adaptation stage consisting of a single gradient update.
... Read more