NB
N. Brouwer
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
5 records found
1
Predictive computational oncology models are fundamentally limited by their uni-modal input drug representations. To overcome this bottleneck, we developed DrugZip, a uniform, task-agnostic, 128-dimensional representation that compresses 25 diverse modalities from the Chemical Checker across a context of 1.2 million molecules. By using a modified autoencoder, DrugZip successfully stabilises the latent space and avoids posterior collapse from a standard variational autoencoder. We evaluated DrugZip across three downstream tasks. In drug synergy prediction, it achieved an AUC of 0.844, resisting performance collapse in unseen cell environments with a mean AUC of 0.62. In drug sensitivity prediction, DrugZip bypassed the extreme overfitting of high-dimensional baselines on unseen drugs. Finally, in cellular perturbation modelling via ChemCPA, DrugZip demonstrated representational sufficiency by matching state-of-the-art transcriptomic prediction accuracy ($R^2$ of 0.776 vs 0.792). Geometrical and information-content analyses confirm that DrugZip produces a continuous, balanced embedding space where drugs remain individually distinguishable. Ultimately, DrugZip shifts the paradigm from engineering task-specific features toward utilising a robust, generalizable, multi-modal representation for computational oncology.
...
Predictive computational oncology models are fundamentally limited by their uni-modal input drug representations. To overcome this bottleneck, we developed DrugZip, a uniform, task-agnostic, 128-dimensional representation that compresses 25 diverse modalities from the Chemical Checker across a context of 1.2 million molecules. By using a modified autoencoder, DrugZip successfully stabilises the latent space and avoids posterior collapse from a standard variational autoencoder. We evaluated DrugZip across three downstream tasks. In drug synergy prediction, it achieved an AUC of 0.844, resisting performance collapse in unseen cell environments with a mean AUC of 0.62. In drug sensitivity prediction, DrugZip bypassed the extreme overfitting of high-dimensional baselines on unseen drugs. Finally, in cellular perturbation modelling via ChemCPA, DrugZip demonstrated representational sufficiency by matching state-of-the-art transcriptomic prediction accuracy ($R^2$ of 0.776 vs 0.792). Geometrical and information-content analyses confirm that DrugZip produces a continuous, balanced embedding space where drugs remain individually distinguishable. Ultimately, DrugZip shifts the paradigm from engineering task-specific features toward utilising a robust, generalizable, multi-modal representation for computational oncology.
Comparative Analysis of Geneformer and Traditional Machine Learning Techniques in Predicting Perturbation Combination Efficacy on Cancer Cell Lines
An Empirical Evaluation Using the sciplex2 Dataset
Cancer poses a significant clinical, social, and economic burden, necessitating the development of effective treatments. Understanding how drugs interact with cancer cells and their downstream effects is critical for creating new therapies and overcoming drug resistance. This paper compares the predictive performance of the Geneformer model with traditional machine learning methods in predicting the response of cancer cells to perturbation combinations using the sciplex2 dataset.
The research involves preprocessing the sciplex2 dataset, training the models, and evaluating their performance in binary classification of cells as either treated or untreated, and the prediction of gene perturbation impacts. While traditional ML models demonstrated higher accuracy in binary classification tasks, Geneformer excelled in predicting the impact of gene perturbations due to its advanced architecture and extensive pre-training on single-cell transcriptomes.
Key findings reveal that highly expression-correlated gene pairs cause the largest shifts in cell classification, underscoring the importance of gene correlations in biological predictions. Geneformer showed a deeper understanding of gene network dynamics, achieving higher maximum Cosine Shifts compared to PCA embeddings and placing less emphasis on highly differentially expressed (HDE) Single Genes. Instead, it focused on HDE Gene Pairs, indicating its potential ability to capture complex downstream effects of gene perturbations.
This study highlights the potential of integrating advanced deep learning models like Geneformer into drug discovery, offering a pathway for more effective and targeted therapeutic interventions. ...
The research involves preprocessing the sciplex2 dataset, training the models, and evaluating their performance in binary classification of cells as either treated or untreated, and the prediction of gene perturbation impacts. While traditional ML models demonstrated higher accuracy in binary classification tasks, Geneformer excelled in predicting the impact of gene perturbations due to its advanced architecture and extensive pre-training on single-cell transcriptomes.
Key findings reveal that highly expression-correlated gene pairs cause the largest shifts in cell classification, underscoring the importance of gene correlations in biological predictions. Geneformer showed a deeper understanding of gene network dynamics, achieving higher maximum Cosine Shifts compared to PCA embeddings and placing less emphasis on highly differentially expressed (HDE) Single Genes. Instead, it focused on HDE Gene Pairs, indicating its potential ability to capture complex downstream effects of gene perturbations.
This study highlights the potential of integrating advanced deep learning models like Geneformer into drug discovery, offering a pathway for more effective and targeted therapeutic interventions. ...
Cancer poses a significant clinical, social, and economic burden, necessitating the development of effective treatments. Understanding how drugs interact with cancer cells and their downstream effects is critical for creating new therapies and overcoming drug resistance. This paper compares the predictive performance of the Geneformer model with traditional machine learning methods in predicting the response of cancer cells to perturbation combinations using the sciplex2 dataset.
The research involves preprocessing the sciplex2 dataset, training the models, and evaluating their performance in binary classification of cells as either treated or untreated, and the prediction of gene perturbation impacts. While traditional ML models demonstrated higher accuracy in binary classification tasks, Geneformer excelled in predicting the impact of gene perturbations due to its advanced architecture and extensive pre-training on single-cell transcriptomes.
Key findings reveal that highly expression-correlated gene pairs cause the largest shifts in cell classification, underscoring the importance of gene correlations in biological predictions. Geneformer showed a deeper understanding of gene network dynamics, achieving higher maximum Cosine Shifts compared to PCA embeddings and placing less emphasis on highly differentially expressed (HDE) Single Genes. Instead, it focused on HDE Gene Pairs, indicating its potential ability to capture complex downstream effects of gene perturbations.
This study highlights the potential of integrating advanced deep learning models like Geneformer into drug discovery, offering a pathway for more effective and targeted therapeutic interventions.
The research involves preprocessing the sciplex2 dataset, training the models, and evaluating their performance in binary classification of cells as either treated or untreated, and the prediction of gene perturbation impacts. While traditional ML models demonstrated higher accuracy in binary classification tasks, Geneformer excelled in predicting the impact of gene perturbations due to its advanced architecture and extensive pre-training on single-cell transcriptomes.
Key findings reveal that highly expression-correlated gene pairs cause the largest shifts in cell classification, underscoring the importance of gene correlations in biological predictions. Geneformer showed a deeper understanding of gene network dynamics, achieving higher maximum Cosine Shifts compared to PCA embeddings and placing less emphasis on highly differentially expressed (HDE) Single Genes. Instead, it focused on HDE Gene Pairs, indicating its potential ability to capture complex downstream effects of gene perturbations.
This study highlights the potential of integrating advanced deep learning models like Geneformer into drug discovery, offering a pathway for more effective and targeted therapeutic interventions.
As a cell, is it better to be single?
Exploring the feasibility of fine-tuning Geneformer on bulk RNA sequencing data
Powerful new machine learning models in biomedicine are being developed constantly, further hastened by the advent of transformer-based architectures. These advanced systems can be used for various applications, from diagnostics to assessing drug effectiveness. Many of these are fundamentally cell classification problems. Models like Geneformer [1] use gene expression data to learn how to distinguish between these cell classes. This information is usually obtained through single-cell RNA sequencing. However, the alternative source, bulk RNA sequencing, offers some advantages that make exploring the feasibility of using it to train Geneformer enticing, such as its greater availability and lower cost. In this paper, pseudo-bulk datasets are created from single-cell data by aggregation of gene expressions. A method to generate synthetic single-cell-like data from a bulk dataset is used to create new datasets. Some remain purely synthetic, while others are mixed with real single-cell data. Geneformer is fine-tuned on all generated datasets separately, and its performance in a cell classification problem is measured. It is shown that the more a dataset resembles real single-cell data, the better the model’s performance. Using bulk data to fine-tune Geneformer is proven to be infeasible. The synthetic data fails to effectively fine-tune the model and is proven to not have a meaningful impact when added to a singlecell dataset. It is concluded that the generated synthetic data is of too low quality and that alternative generation methods should be explored.
...
Powerful new machine learning models in biomedicine are being developed constantly, further hastened by the advent of transformer-based architectures. These advanced systems can be used for various applications, from diagnostics to assessing drug effectiveness. Many of these are fundamentally cell classification problems. Models like Geneformer [1] use gene expression data to learn how to distinguish between these cell classes. This information is usually obtained through single-cell RNA sequencing. However, the alternative source, bulk RNA sequencing, offers some advantages that make exploring the feasibility of using it to train Geneformer enticing, such as its greater availability and lower cost. In this paper, pseudo-bulk datasets are created from single-cell data by aggregation of gene expressions. A method to generate synthetic single-cell-like data from a bulk dataset is used to create new datasets. Some remain purely synthetic, while others are mixed with real single-cell data. Geneformer is fine-tuned on all generated datasets separately, and its performance in a cell classification problem is measured. It is shown that the more a dataset resembles real single-cell data, the better the model’s performance. Using bulk data to fine-tune Geneformer is proven to be infeasible. The synthetic data fails to effectively fine-tune the model and is proven to not have a meaningful impact when added to a singlecell dataset. It is concluded that the generated synthetic data is of too low quality and that alternative generation methods should be explored.
Strategies for Fine-Tuning Geneformer to Predict the Exposure Level of Cancer Cells to Treatments
A Comparison of Different Fine-Tuning Strategies for Foundation Models
Studying the interactions of genes within a cell is an area of significant interest in the field of medicine as it can provide answers to what exactly drives the behavior of a cell under specific circumstances, such as diseases. Once understood, gene interactions can enable the synthesis of efficient, possibly personalized treatments for these diseases and other disorders. However, studying gene interactions requires a large number of samples which might be costly and laborious to obtain in the case of rare disorders for which there is not much recorded data. Geneformer, a context-aware, attention-based deep-learning model, was created specifically for solving this problem. The model makes use of transfer learning to apply any relevant knowledge gained from a larger, similar domain onto a downstream domain with limited data which can be used to further train the model. In this paper, we assessed four fine-tuning strategies, including the one used throughout the in silico experiments presented in the original Geneformer paper. We did this to assess whether the accuracy of Geneformer on the downstream task of predicting the sensitivity of cancer cells to different treatments can be improved versus the default implementation as found within the model's paper. The model was firstly fine-tuned using a training dataset compiled from the sciplex2 dataset, followed by the prediction of the dosage levels to which samples from a test set were exposed. Upon performing the experiment, we concluded that, depending on the way in which knowledge from the source domain is stored inside the pre-trained model and the similarity between the source and the target domains, different fine-tuning strategies were suitable for a given task. Hence, there is no single optimal fine-tuning method which can be used to predict the level to which cancer cells were exposed to treatments such as nutlin-3A.
...
Studying the interactions of genes within a cell is an area of significant interest in the field of medicine as it can provide answers to what exactly drives the behavior of a cell under specific circumstances, such as diseases. Once understood, gene interactions can enable the synthesis of efficient, possibly personalized treatments for these diseases and other disorders. However, studying gene interactions requires a large number of samples which might be costly and laborious to obtain in the case of rare disorders for which there is not much recorded data. Geneformer, a context-aware, attention-based deep-learning model, was created specifically for solving this problem. The model makes use of transfer learning to apply any relevant knowledge gained from a larger, similar domain onto a downstream domain with limited data which can be used to further train the model. In this paper, we assessed four fine-tuning strategies, including the one used throughout the in silico experiments presented in the original Geneformer paper. We did this to assess whether the accuracy of Geneformer on the downstream task of predicting the sensitivity of cancer cells to different treatments can be improved versus the default implementation as found within the model's paper. The model was firstly fine-tuned using a training dataset compiled from the sciplex2 dataset, followed by the prediction of the dosage levels to which samples from a test set were exposed. Upon performing the experiment, we concluded that, depending on the way in which knowledge from the source domain is stored inside the pre-trained model and the similarity between the source and the target domains, different fine-tuning strategies were suitable for a given task. Hence, there is no single optimal fine-tuning method which can be used to predict the level to which cancer cells were exposed to treatments such as nutlin-3A.
Evaluating Machine Learning Approaches for Predicting Drug Response in Cancer Cells
A Comparative Analysis of Geneformer and Support Vector Machine
Accurately predicting how cancer cells respond to drug treatment is important to advance drug development. This paper presents a comparative analysis of Geneformer, a deep-learning transformer pre-trained on transcriptomic data, and Support Vector Machine. Using the Sciplex2 dataset, which includes transcriptomic data from lung cancer cells treated with three drugs, both models were trained to predict the response of cancer cells to drug treatments.
This paper investigates how Geneformer and SVM perform in predicting the treatment label of cells across different drugs and doses, which drug doses are suitable for conducting single-gene perturbation experiments, how accurately can these experiments replicate drug effects, and what are the differences in results between Geneformer and SVM regarding their ability to identify significant genes affecting drug response.
Results indicate that while SVM generally achieves higher accuracy in predicting treatment labels of cells, Geneformer demonstrates better capability in identifying genes whose perturbations mimic drug effects. Geneformer's embeddings show significant shifts towards treated cell states after single-gene perturbations, indicating a deeper understanding of gene interactions in drug response. On the other hand, SVM's predictions rely more on differential gene expression. This comparative analysis underscores the strengths and limitations of each approach in modelling complex biological systems and predicting the drug response of cancer cells. ...
This paper investigates how Geneformer and SVM perform in predicting the treatment label of cells across different drugs and doses, which drug doses are suitable for conducting single-gene perturbation experiments, how accurately can these experiments replicate drug effects, and what are the differences in results between Geneformer and SVM regarding their ability to identify significant genes affecting drug response.
Results indicate that while SVM generally achieves higher accuracy in predicting treatment labels of cells, Geneformer demonstrates better capability in identifying genes whose perturbations mimic drug effects. Geneformer's embeddings show significant shifts towards treated cell states after single-gene perturbations, indicating a deeper understanding of gene interactions in drug response. On the other hand, SVM's predictions rely more on differential gene expression. This comparative analysis underscores the strengths and limitations of each approach in modelling complex biological systems and predicting the drug response of cancer cells. ...
Accurately predicting how cancer cells respond to drug treatment is important to advance drug development. This paper presents a comparative analysis of Geneformer, a deep-learning transformer pre-trained on transcriptomic data, and Support Vector Machine. Using the Sciplex2 dataset, which includes transcriptomic data from lung cancer cells treated with three drugs, both models were trained to predict the response of cancer cells to drug treatments.
This paper investigates how Geneformer and SVM perform in predicting the treatment label of cells across different drugs and doses, which drug doses are suitable for conducting single-gene perturbation experiments, how accurately can these experiments replicate drug effects, and what are the differences in results between Geneformer and SVM regarding their ability to identify significant genes affecting drug response.
Results indicate that while SVM generally achieves higher accuracy in predicting treatment labels of cells, Geneformer demonstrates better capability in identifying genes whose perturbations mimic drug effects. Geneformer's embeddings show significant shifts towards treated cell states after single-gene perturbations, indicating a deeper understanding of gene interactions in drug response. On the other hand, SVM's predictions rely more on differential gene expression. This comparative analysis underscores the strengths and limitations of each approach in modelling complex biological systems and predicting the drug response of cancer cells.
This paper investigates how Geneformer and SVM perform in predicting the treatment label of cells across different drugs and doses, which drug doses are suitable for conducting single-gene perturbation experiments, how accurately can these experiments replicate drug effects, and what are the differences in results between Geneformer and SVM regarding their ability to identify significant genes affecting drug response.
Results indicate that while SVM generally achieves higher accuracy in predicting treatment labels of cells, Geneformer demonstrates better capability in identifying genes whose perturbations mimic drug effects. Geneformer's embeddings show significant shifts towards treated cell states after single-gene perturbations, indicating a deeper understanding of gene interactions in drug response. On the other hand, SVM's predictions rely more on differential gene expression. This comparative analysis underscores the strengths and limitations of each approach in modelling complex biological systems and predicting the drug response of cancer cells.