Studying the interactions of genes within a cell is an area of significant interest in the field of medicine as it can provide answers to what exactly drives the behavior of a cell under specific circumstances, such as diseases. Once understood, gene interactions can enable the synthesis of efficient, possibly personalized treatments for these diseases and other disorders. However, studying gene interactions requires a large number of samples which might be costly and laborious to obtain in the case of rare disorders for which there is not much recorded data. Geneformer, a context-aware, attention-based deep-learning model, was created specifically for solving this problem. The model makes use of transfer learning to apply any relevant knowledge gained from a larger, similar domain onto a downstream domain with limited data which can be used to further train the model. In this paper, we assessed four fine-tuning strategies, including the one used throughout the in silico experiments presented in the original Geneformer paper. We did this to assess whether the accuracy of Geneformer on the downstream task of predicting the sensitivity of cancer cells to different treatments can be improved versus the default implementation as found within the model's paper. The model was firstly fine-tuned using a training dataset compiled from the sciplex2 dataset, followed by the prediction of the dosage levels to which samples from a test set were exposed. Upon performing the experiment, we concluded that, depending on the way in which knowledge from the source domain is stored inside the pre-trained model and the similarity between the source and the target domains, different fine-tuning strategies were suitable for a given task. Hence, there is no single optimal fine-tuning method which can be used to predict the level to which cancer cells were exposed to treatments such as nutlin-3A.

Powerful new machine learning models in biomedicine are being developed constantly, further hastened by the advent of transformer-based architectures. These advanced systems can be used for various applications, from diagnostics to assessing drug effectiveness. Many of these are fundamentally cell classification problems. Models like Geneformer [1] use gene expression data to learn how to distinguish between these cell classes. This information is usually obtained through single-cell RNA sequencing. However, the alternative source, bulk RNA sequencing, offers some advantages that make exploring the feasibility of using it to train Geneformer enticing, such as its greater availability and lower cost. In this paper, pseudo-bulk datasets are created from single-cell data by aggregation of gene expressions. A method to generate synthetic single-cell-like data from a bulk dataset is used to create new datasets. Some remain purely synthetic, while others are mixed with real single-cell data. Geneformer is fine-tuned on all generated datasets separately, and its performance in a cell classification problem is measured. It is shown that the more a dataset resembles real single-cell data, the better the model’s performance. Using bulk data to fine-tune Geneformer is proven to be infeasible. The synthetic data fails to effectively fine-tune the model and is proven to not have a meaningful impact when added to a singlecell dataset. It is concluded that the generated synthetic data is of too low quality and that alternative generation methods should be explored.

Accurately predicting how cancer cells respond to drug treatment is important to advance drug development. This paper presents a comparative analysis of Geneformer, a deep-learning transformer pre-trained on transcriptomic data, and Support Vector Machine. Using the Sciplex2 dataset, which includes transcriptomic data from lung cancer cells treated with three drugs, both models were trained to predict the response of cancer cells to drug treatments. This paper investigates how Geneformer and SVM perform in predicting the treatment label of cells across different drugs and doses, which drug doses are suitable for conducting single-gene perturbation experiments, how accurately can these experiments replicate drug effects, and what are the differences in results between Geneformer and SVM regarding their ability to identify significant genes affecting drug response. Results indicate that while SVM generally achieves higher accuracy in predicting treatment labels of cells, Geneformer demonstrates better capability in identifying genes whose perturbations mimic drug effects. Geneformer's embeddings show significant shifts towards treated cell states after single-gene perturbations, indicating a deeper understanding of gene interactions in drug response. On the other hand, SVM's predictions rely more on differential gene expression. This comparative analysis underscores the strengths and limitations of each approach in modelling complex biological systems and predicting the drug response of cancer cells.

This paper presents a novel approach to measuring bias in Automatic Speech Recognition (ASR) systems by proposing a metric that does not use the conventional approach of a reference group. Current methods typically measure bias through comparison with a ’norm’ or minimum error group, potentially introducing additional biases. To address this issue, this study introduces a new metric: a combination of the Groupto-Average Log Ratio and the Sum of Group Error Differences. This metric aims to provide a fair comparison by measuring performance relative to the average of the groups rather than a single reference group. Results indicate that the new metric reveals different aspects of bias not captured by traditional methods. This study contributes to the ongoing research on fairness in speech technology by challenging the existing bias metrics and proposing alternatives that might offer more equitable evaluations. Future research should explore further refinements of these metrics, and apply them across more varied datasets and environments. Ultimately, this research moves towards making ASR technologies more inclusive, ensuring that they serve all user groups equitably.

Automatic Speech Recognition (ASR) systems have become an integral part of daily lives. Despite their widespread use, these systems can exhibit biases that express themselves in the differences in their accuracy and performance across different demographic groups. Methods quantifying these biases have been developed. This paper investigates the relationship between bias and the acoustic characteristics of speakers. By examining various acoustic embeddings, derived from models like wav2vec 2.0 and XLSR, we aim to identify which embeddings correlate most strongly with bias. The findings offer insights into improving the fairness of ASRs by exploring how acoustic features influence bias in ASR systems. Future research directions include exploring isolated speech properties and extending the study to diverse linguistic contexts to deepen understanding in this area.

Cancer poses a significant clinical, social, and economic burden, necessitating the development of effective treatments. Understanding how drugs interact with cancer cells and their downstream effects is critical for creating new therapies and overcoming drug resistance. This paper compares the predictive performance of the Geneformer model with traditional machine learning methods in predicting the response of cancer cells to perturbation combinations using the sciplex2 dataset. The research involves preprocessing the sciplex2 dataset, training the models, and evaluating their performance in binary classification of cells as either treated or untreated, and the prediction of gene perturbation impacts. While traditional ML models demonstrated higher accuracy in binary classification tasks, Geneformer excelled in predicting the impact of gene perturbations due to its advanced architecture and extensive pre-training on single-cell transcriptomes. Key findings reveal that highly expression-correlated gene pairs cause the largest shifts in cell classification, underscoring the importance of gene correlations in biological predictions. Geneformer showed a deeper understanding of gene network dynamics, achieving higher maximum Cosine Shifts compared to PCA embeddings and placing less emphasis on highly differentially expressed (HDE) Single Genes. Instead, it focused on HDE Gene Pairs, indicating its potential ability to capture complex downstream effects of gene perturbations. This study highlights the potential of integrating advanced deep learning models like Geneformer into drug discovery, offering a pathway for more effective and targeted therapeutic interventions.

Anesthesia-related hypotension is a significant concern during surgery, occurring shortly after induction and potentially leading to severe complications. Since the anesthetic drug is believed to have an important role in the occurrence of post-induction hypotension (PIH), anesthesiologists now advocate for the appropriate selection of anesthetics dosage to avoid PIH.To facilitate such decision-making, an accurate prediction of PIH associated with a certain dosage of anesthetics is necessary. This thesis presents a high-accuracy prediction model for PIH that supports anesthesia decision-making. The model is trained on data from the VitalDB database of 320 patients undergoing general anesthesia. The target output of this classification model is the occurrence of PIH, as defined through comprehensive analysis that incorporates clinical operations. Besides demographic data and vital signs, our model incorporates the dosage of propofol administered during the induction period as an input variable, mimicking real-world anesthetic plans. By employing the model in the target control infusion system of anesthesia, the anesthetics dosage can be varied as input, providing outcome predictions as security suggestions. An ensemble algorithm is employed to balance the prediction performance and the ability to elucidate the positive relationship between propofol and PIH risk, forming an anesthetics advice model. Compared to previous PIH prediction studies, our prediction model is validated in more reliable nested cross-validation approach and achieves a higher performance (precision of 0.83 and recall of 0.84). We believe utilizing demographic and dynamic vital signs to predict HIP can be useful in determining the appropriate anesthetic dosage plan, offering potential improvements in patient care and safety.

The advancement of artificial intelligence (AI) has led to an increased demand for both a greater volume and quality of data. In many companies, data is dispersed across multiple tables, yet AI models typically require data in a single table format. This necessitates the merging of these tables and the selection of optimal features for the model, a process known as Tabular Data Augmentation (TDA). With the rapid growth of TDA, automated tools have been developed to streamline this process. However, these state-of-the-art tools often make assumptions about user workflows that may not align with the actual needs of data specialists, potentially making them efficient yet not fully user-friendly. Additionally, without thorough evaluation through user studies, these tools may overlook critical steps in the TDA process. This thesis is divided into two main parts. The first part is dedicated to uncovering the assumptions and oversights within current TDA research through an exhaustive review of recent literature. This is followed by conducting interviews with 19 data specialists. These discussions aim to verify the identified assumptions and reveal any missing elements in state-of-the-art research. The second part focuses on creating a new tool to meet the requirements identified from validated assumptions and the gaps discovered. This tool is then subjected to evaluation interviews to assess its effectiveness. The findings indicate that data specialists prefer a TDA tool that offers enhanced control and deeper insights into the data augmentation process. To meet these preferences, Human in the Loop AutoTDA was developed, embodying the desired functionalities. Feedback from the evaluation phase confirmed that data specialists find Human in the Loop AutoTDA suitable for their TDA workflows, marking a significant advancement in the field.

In recent advancements within the field of machine learning (ML), the automation of model development and deployment enabled the maintenance of high-quality models in production through continuous retraining, yielding a variety of models for the same problem settings. The fast moving progress of Large-Language-Models and other research areas let to a spiking interest in ML, expanding applications and the market for ML models. Researchers, enthusiasts, programming frameworks and major companies such as Amazon are actively developing and evaluating models for a wide variety of tasks, thus contributing to the growing number of available pretrained models. These models are accessible through various platforms, including Huggingface or the AWS Sagemaker program. The most common cause for retraining of models in production is a distribution shift in the data. Consequently, continuous retraining on changing production data results in a wide variety of models addressing the same problem, each with different strengths. Each retraining iteration demands substantial amounts of labeled data and significant time investment. Given the increasing availability of pretrained models with distinct strengths and weaknesses, there is strong reason to believe that, instead of retraining, selecting the most suitable pretrained model can yield sufficient performance. The primary challenge in the context of model selection is the need for evidence to assess the quality of the models, and querying labels is the most effective method to address this. Despite the progress in ML research, the acquisition of labeled data remains a significant challenge. Labeling data is mostly achieved through human labor, which is a costly, time consuming, and error-prone process. Estimates based on AWS Mechnical Turk indicate that the expenses of labeling large datasets or complex tasks, such as image segmentation, can easily reach six figures or more. For instance, labeling the entire ImageNet dataset with ten workers costs approximately 190.000€. This necessitates minimizing the number of labels required as evidence for model selection. The existing literature on model selection primarily focuses on selection of a learning strategy in combination with its optimal hyperparameters to best fit the data. Although the objective of selecting the best model for the data remains the same, the setting in this thesis differs significantly. Specifically, the architecture and hyperparameters of the models are not relevant in this research, as they are predefined by the pretrained candidate models. Furthermore, the data used to assess the models is unlabeled, leading to a discussion on the suitability of alternative methods more closely aligned with this setting, such as active learning (AL), which focuses on intelligently labeling data points for training. Given the lack of appropriate methods for this problem, this thesis introduces the Model Picker algorithm as a solution for selecting pretrained classifiers. The algorithm aims to minimize labeling cost by employing a probabilistic model to adaptively sample the most valuable instances from the unlabeled data. The informativeness of a data point is estimated using Shannon’s mutual information between the latent variable, which represents the decision about the true best model, and the unknown labels of each data point, given the evidence sampled. The Model Picker algorithm was rigorously evaluated against fundamental baselines such as Query-by-Committee, Active Comparison of Prediction Models, GALAXY, and QDD which were adapted to the setting when necessary. This evaluation utilizes a wide variety of well-established datasets, including up to 114 of different models each for each dataset, such as ImageNet. The results demonstrate that Model Picker consistently outperforms all baselines by a significant margin, with peak performance that allows for accurate model selection with only 1/4 of the labels needed compared to the next best method. Additionally, this thesis investigates various methods for efficient optimization of Model Picker’s single hyperparameter, ϵ, including strategies such as sampling a small subset of data for labeling and generating a noisy oracle. The proposed framework is intended to serve as a fundamental stepping stone for future research in the domain of model selection. The thesis concludes with a discussion on the implications of the findings for the Model Picker algorithm, as well as for future research in the selection of pretrained classifiers. Furthermore, limitations of this study are discussed and potential future work to further enhance the Model Picker algorithm is proposed.

Multiple benchmarks for question answering (QA) systems often under-represent questions that require lists to be answered, referred to in this work as ListQA. This type of question can provide valuable insights into the system’s ability to structure its internal knowledge. In this work, we introduce TimelineQA (TLQA), a specialized subset of ListQA that requires both list comprehension and temporal understanding to generate correct answers. We automatically curate a benchmark for TLQA and test it against generative large language models (LLMs) commonly employed in QA systems. Our findings reveal significant shortcomings in current models, particularly in their inability to provide complete answers and accurately align facts temporally. To address these issues, we explore how Retrieval Augmented Generation (RAG) can enhance performance in this task. Our results indicate that while RAG improves performance, especially in retrieving relevant contextual information, there remains considerable room for further improvement. Both the benchmark and the evaluation of the models can be publicly accessed here.

Automatic Speech Recognition (ASR) systems are becoming increasingly popular in this day and age. Unfortunately, due to inherent biases within these systems, performance disparities exist among specific demographic groups. Bias metrics can be used to measure this bias. Within ASR they represent a niche area that has not yet been thoroughly explored. The few bias metrics that exist in literature mainly centre around the performance differences between speaker groups. This paper proposes two new bias metrics that focus not only on performance differences, but also take the base performance into account: Weighted Performance Bias (WPB) and Intergroup Weighted Performance Bias (IWPB). Although the lack of ground truth makes the results less easily interpretable, the results show similar trends within the new metrics as those defined in literature: bias is greatest among non-native Dutch speech.

Dutch State-of-the-art Automatic Speech Recognition (ASR) systems do not perform equally well for different speaker groups. Existing metrics to quantify this bias rely on demographic metadata, which is often unavailable. Recent advances in the field use machine learning to find groups of similar speakers instead. However, its black-box nature obscures the interpretability of resulting groups. This paper proposes an interpretable approach to bias discovery by clustering speakers based on acoustic and prosodic features. Different feature subsets were compared in their ability to find performance disparities in five ASR systems for two separate speaking styles. Results show that these feature sets can uncover bias approaching known disparities between demographic groups. While the effectiveness per feature set differed between the speaking styles, the most successful ones found significant disparities between clusters with diverse demographic compositions.