Support Vector Machines (SVMs) are widely used in various domains, with their performance heavily dependent on hyperparameter selection. However, hyperparameter tuning is computationally demanding due to the SVM training complexity, which is at best $O(n^2)$, where $n$ represents the number of training samples. To mitigate this challenge, we propose integrating a validation-based early stopping criterion into the Sequential Minimal Optimization (SMO) algorithm to enhance tuning efficiency.

We evaluate this approach within Random Search and Successive Halving frameworks, aiming to reduce tuning runtime while preserving model performance. We introduce a composite score function to facilitate a balanced assessment of accuracy and efficiency. Our empirical analysis reveals that incorporating early stopping into SMO significantly reduces hyperparameter tuning time under RS but provides limited benefits in successive halving, given its inherent efficiency. Additionally, while dataset characteristics influence the effectiveness of early stopping, we found evidence that dimensionality does not. We also observe that frequent early stopping objective assessments introduce computational overhead, which can offset runtime improvements. Reducing assessment frequency alleviates this issue, but it diminishes the effectiveness of early stopping.

Our findings highlight the potential of early stopping in SMO for optimizing SVM hyperparameter tuning, particularly within random search-based approaches. They also identify trade-offs in assessment frequency and dataset-specific factors.

Stroke remains a leading cause of morbidity and mortality worldwide, despite advances in treatment modalities. Endovascular thrombectomy (EVT), a revolutionary intervention for ischemic stroke, is limited by its reliance on 2D fluoroscopic imaging, which lacks depth and comprehensive vascular detail. We propose a novel AI-driven pipeline for 3D CTA to 2D DSA cross-modality registration, termed DeepIterReg. The proposed pipeline integrates neural network-based initialization with iterative optimization to align pre-intervention and peri-intervention data. Our approach addresses the challenges of cross-modality alignment, particularly in scenarios involving limited shared vascular structures, by leveraging synthetic data, vein-centric anchoring, and differentiable rendering techniques. We assess the efficacy of DeepIterReg through quantitative analysis of capture ranges and registration accuracy. Results indicate that our method is able to accurately register 70% of a testset of 20 patients, and is able to improve capture ranges when performing an initial pose estimation using a convolutional neural network.

Synthetic polymers are crucial in diverse industries, but current AI-driven design methodologies primarily target linear homopolymers, with limited emphasis on developing customized approaches for copolymers. To address this gap, we introduce a generative model for goal-directed synthetic copolymer design using reinforcement learning. Our model operates in a graph generation environment, facilitating efficient monomer unit design while incorporating domain-specific constraints to ensure high validity rates. In a case study optimizing for Hydrogen Evolution Rate (HER) and synthetic accessibility, our approach showcases the efficacy of reinforcement learning in advancing copolymer design. Furthermore, experimental results underscore the challenges in designing effective scoring functions due to the sparse nature of polymer datasets, emphasizing the need for robust property predictors in polymer design methodologies before integrating more complex generative models into the design process.

Physics-Informed Neural Networks (PINNs) offer a promising approach to solving partial differential
equations (PDEs). In PINNs, physical laws are incorporated into the loss function, guiding the network to learn a model that adheres to these laws as defined by the PDEs. Training PINNs involves using three types of points: collocation points within the domain of the PDEs, boundary points on the edges of the domain, and initial points at the starting time (t = 0). A common challenge in training PINNs is the imbalance between the number of boundaries and initial points compared to collocation points, which can negatively impact the training process. Typically, the number of each type of point is manually set, and the number of collocation points is usually ten times that of boundary and initial points, leading to an uneven loss distribution over time. Our work introduces a method called Relative Residual Resampling (R3) to address this issue. This method dynamically adjusts the number of each type of training point to ensure a more balanced distribution. Consequently, the loss is more evenly spread across the time domain, enhancing the performance of the PINN. We tested our method on the 1D Heat Equation and 1D Diffusion Equation. The results demonstrate that our approach reduces the overall loss by at least 6%, and balances the loss distribution over time by reducing the loss range for at least 65% compared with the state-of-the-art residual-based resampling strategy. This improvement makes PINNs more accurate and efficient for solving time-dependent PDEs, providing a practical solution for applications where understanding temporal dynamics is crucial.

Recent advancements in machine learning (ML) have shown promise in accelerating polymer discovery by aiding in tasks such as virtual screening via property prediction, and the design of new polymer materials with desired chemical properties. However, progress in polymer ML is hampered by the scarcity of high-quality, labelled datasets, which are necessary for training supervised ML models. In this work, we study the use of the very recent ’Joint Embedding Predictive Architecture’ (JEPA) type for self-supervised learning (SSL) on polymer molecular graphs, to understand whether pretraining with the proposed SSL strategy improves downstream performance when labelled data is scarce. By doing so, this study aims to shed light on this new family of architectures in the molecular graph domain and provide insights and directions for future research on JEPAs. Our experimental results indicate that JEPA self-supervised pretraining enhances downstream performance, particularly when labelled data is very scarce, achieving improvements across all tested datasets.

The emergence of Language Language Models (LLMs)-based agents represents a significant advancement in artificial intelligence (AI), offering new possibilities for complex problem-solving and interaction within a virtual environment. Our work is based on the Voyager paper [1], which is a state-of-the-art LLM-based agent for Minecraft. However, this system suffers from some significant limitations, such as its reliance on closed-source LLMs and lack of social awareness. Indeed, current open-source LLMs often fail to match closed-source ones in the agent setting, leaving research reliant on third-party closed-source technology [1] [2]. This gap highlights the need for alternative strategies to enhance LLM performance without the high costs associated with fine-tuning. To address these challenges, we propose the Collaborative Voyager, a new architecture designed to enable agent collaboration and social awareness using open-source LLMs. Inspired by the social intelligence hypothesis, which suggests that intelligence emerges from social interactions, we propose collaboration as an alternative learning paradigm for LLMs. This alternative learning paradigm could potentially supplement the expensive fine-tuning currently needed to bridge the performance gap between open-source and closed-source models in the agent setting [2]. Our approach involves developing a framework that allows agents to communicate, understand, and learn from each other, enabling them to correct errors and adapt to new tasks dynamically. By using a memory module, our agent is able to remember interactions and learn from them in order to accomplish a task that it was previously unable to do on its own. Through various experiments, we demonstrate that collaboration significantly enhances the performance of LLM agents in both task completion and adaptability, addressing issues like hallucinations. This study provides insights into developing more sophisticated, adaptable AI systems capable of dynamic interactions and problem-solving. These findings have potential applications extending beyond Minecraft and virtual environments to fields such as robotics, where collaboration and social awareness are crucial.

Cancer is one of the leading causes of death. To reduce the amount of deaths caused by cancer, a number of different screening methods are used to detect cancer in an earlier stage, to improve sur vival rates when treating patients with cancer. Cur rent screening methods are often invasive, costly and not very accurate. Therefore, new methods are being sought that aim to be cheaper, less in vasive and provide more accurate results. One of these methods is fragmentomics. Multiple methods have been proposed to use fragmentomics analy sis in the context of screening for cancer, includ ing using the short/long ratio as well as investigat ing the nucleotides at the ends of the fragments. Across previous works using fragmentomics anal ysis to predict cancer, different pre-proccessing steps are used, with limited explanation why the pre-processing methods were chosen. Research into the effects of pre-processing steps used when using fragmentomics analysis is lacking. Two main pre-processing steps in the field are correct ing GC-bias and filtering on MAPQ. Here we in vestigated the impact of three GC-correction meth ods by applying the correction method and then analyzing the resulting fragmentation profiles us ing short/long fragment ratios. Furthermore, three different MAPQ filtering thresholds were studied. This showed that Deeptools correction of the GC bias lowered performance, with the accuracy drop ping from 77.8% to 69.4%. Applying LOESS cor rection using all fragments at the same time re sulted in an accuracy of 83.3%, while applying LOESS correction using the short and long frag ments separately resulted in an accuracy of 91.7%. The impact of filtering the data based on mapping quality was determined by comparing the results of analysing all fragments, analyzing only fragments with mapping quality 5, 20 or 30. This showed that not filtering by mapping quality has a big impact on the profiles of cancer samples, with a KS-test statistic of 0.08 for MAPQ 5 and MAPQ 20 and larger differences in correlations between healthy and cancer samples. The performance of classi fication was much higher when not filtering, with an accuracy of 97.3%, which dropped whenever the filtering threshold was raised, bottoming out at 62.7% for a threshold of MAPQ 30. Due to limita tions with the study, the combined pre-processing of not filtering on MAPQ and using the LOESS separate correction were not studied.

The field of causal inference provides a variety of estimators that can be used to find the effect of a treatment on an outcome based on observational data. However, many of these estimators require the unconfoundedness assumption, stating that all relevant confounders are observed within the data. This assumption is quite strict and many real-life problems would violate it, due to confounders being too expensive, immoral, or abstract to observe. To weaken this assumption, causal sensitivity analysis attempts to quantify the level of hidden confounding and use it to create a possible bound on the causal effect. This study compares a well-established Marginal Sensitivity Model (MSM) with a newly proposed f-sensitivity model. Given f-sensitivity is relatively new, the current approaches for computation can be inefficient or non-deterministic. A new method of computing the f-sensitivity bound is proposed, which can lead to closed-form solutions in some estimator-specific cases. The differences between the MSM and f-sensitivity models are outlined and illustrated with examples, from which a set of guiding questions is created for researchers to decide between the two sensitivity models. All of the code required to reproduce this study is located in this GitHub repository.

Proteins are fundamental biological macromolecules essential for cellular structure, enzymatic catalysis, and immune defense, making the generation of novel proteins crucial for advancements in medicine, biotechnology, and material sciences. This study explores protein design using deep generative models, specifically Denoising Diffusion Probabilistic Models (DDPMs). While traditional methods often focus on either protein structure or sequence design independently, recent trends emphasize a co-design approach addressing both aspects simultaneously. We propose a novel methodology utilizing Equivariant Graph Neural Networks (EGNNs) within the diffusion framework to co-design protein structures and sequences. We modify the EGNN architecture to improve its effectiveness in learning intricate data patterns. Experimental results show that our approach effectively generates high-quality protein sequences, although challenges remain in producing plausible protein backbones and ensuring strong sequence-structure correlation.

Studying the interactions of genes within a cell is an area of significant interest in the field of medicine as it can provide answers to what exactly drives the behavior of a cell under specific circumstances, such as diseases. Once understood, gene interactions can enable the synthesis of efficient, possibly personalized treatments for these diseases and other disorders. However, studying gene interactions requires a large number of samples which might be costly and laborious to obtain in the case of rare disorders for which there is not much recorded data. Geneformer, a context-aware, attention-based deep-learning model, was created specifically for solving this problem. The model makes use of transfer learning to apply any relevant knowledge gained from a larger, similar domain onto a downstream domain with limited data which can be used to further train the model. In this paper, we assessed four fine-tuning strategies, including the one used throughout the in silico experiments presented in the original Geneformer paper. We did this to assess whether the accuracy of Geneformer on the downstream task of predicting the sensitivity of cancer cells to different treatments can be improved versus the default implementation as found within the model's paper. The model was firstly fine-tuned using a training dataset compiled from the sciplex2 dataset, followed by the prediction of the dosage levels to which samples from a test set were exposed. Upon performing the experiment, we concluded that, depending on the way in which knowledge from the source domain is stored inside the pre-trained model and the similarity between the source and the target domains, different fine-tuning strategies were suitable for a given task. Hence, there is no single optimal fine-tuning method which can be used to predict the level to which cancer cells were exposed to treatments such as nutlin-3A.

This paper describes ZygosDB, a novel and efficient read-only database designed specifically for querying positional genomic data required for Genome-Wide Association Studies (GWAS). ZygosDB addresses limitations of existing solutions like Tabix by offering optimized data structures, compression techniques, and parallel query execution.

Our evaluation shows that ZygosDB achieves a significant speedup of 2-5 times in query throughput compared to Tabix. This improvement comes from our focus on efficient data storage and retrieval tailored to the specific needs of GWAS.

The paper also explores the impact of multi-threading on query performance and the role of compression algorithms in optimizing query throughput. We identify a decrease in performance beyond a certain number of threads and discuss the influence of compression algorithms like Gzip and LZ4.

While ZygosDB offers substantial performance gains, future work should explore avenues for further optimization, such as measuring query latency, refining memory usage, and investigating specialized column support. Overall, ZygosDB establishes itself as a powerful tool for efficient querying of large genomic datasets, facilitating more effective GWAS.

Individualizing mechanical ventilation treatment regimes remains a challenge in the intensive care unit (ICU). Reinforcement Learning (RL) offers the potential to improve patient outcomes and reduce mortality risk, by optimizing ventilation treatment regimes. We focus on the Offline RL setting, using Offline Policy Evaluation (OPE), specifically importance sampling (IS), to evaluate policies learned from observational data. Using a running example, we illustrate how a large difference between the learned policy and actual clinical behavior (behavior policy) limits the reliability of IS-based OPE. To assess this reliability, we use the Effective Sample Size (ESS) as a diagnostic. To achieve reliable evaluation, we apply policy shaping, by incorporating a divergence constraint in the policy learning objective, aiming to reduce the difference between the evaluation and behavior policy. We consider both a Kullback-Leibler (KL) divergence constraint and introduce a new constraint, the ESS divergence. Since effective OPE relies on an accurate estimate of the true behavior policy, we address how such an estimate is acquired. Various classifiers for estimating the behavior policy are systematically evaluated, focusing on both discrimination and calibration performance. Empirical results show the difficulty of learning policies that outperform existing clinical practices and generalize well to unseen patients. Although policy shaping improves the reliability of policy evaluations, no policies that consistently outperform clinician practice were found. The KL divergence constraint generalized better to unseen patients than the ESS divergence, which achieved large ESS without actually reducing the difference between the evaluation and behavior policy. We underscore the necessity of a cautious approach to applying RL in healthcare, and advocate that assessing OPE reliability and behavior policy calibration becomes standard practice, to ensure that only effective and reliable RL policies are considered for real-world clinical trials.

The objective of this project is to train a model that transforms a tree with its foliage into only its branch structure. This is achieved by employing machine-learning techniques, specifically Generative Adverserial Networks (GANs). By utilizing the proposed method, a predictive model is built that automatically minimizes its own error function through a comparison of a set of input and ground-truth tree images, which are tree images with and without leaves, respectively. The adoption of GANs has shown promising results, both visually and metrically.

L-Systems allow for the efficient procedeural generation of trees to be used for rendering in video games and simulations. Currently, however, it is difficult to engineer grammars that mimic the behaviours of real life trees in 3 dimensions. To be able to deduce them, the skeleton of a tree can be used to train a model and generate an L-system for a given tree in particular. The aim of this paper is to provide a pipeline to isolate these skeletons from images of a tree, using Neural Radiance Fields (NeRFs) to reconstruct the tree, and using Laplacian Based Contraction to retrieve the underlying skeleton. We find that this approach leads to 3-dimensional topologies that very closely resemble the given tree.

Trees are essential components of both real and digital environments. Therefore, it is important to have 3D models of trees that are of high quality and computationally efficient. One way to achieve this is by compressing a high-quality model using billboard rendering, which involves partitioning the tree into multiple planes to produce a similar result to the original. Our study explores the compression of 3D models using an optimization loop and adapting billboard rendering techniques. We use computer vision primitives to render basic models, which we then optimize by adjusting the texture to resemble the original tree. The models consist of multiple upright planes that are rotated around the central vertical axis of the original tree. We use different optimization functions, such as L1 and L2 losses, to determine the best approach. We can improve the initial models by bounding the billboards and limiting their heights and widths to that of the trees. Additionally, we can use double-sided textures for the billboards to allow more flexibility for optimizing different species of trees. However, optimizing multiple tree types performs differently for each species, leading to improvements that only benefit certain trees in specific scenarios. Using quantitative metrics, we determined which models perform best and how similar they are to the original after training. We found that our compressed models generally resemble the original while having only a fraction of the original size.

There exists a fundamental gap between human and artificial intelligence. Deep learning models are exceedingly data hungry for learning even the simplest of tasks, whereas humans can easily adapt to new tasks with just a handful of samples. Unsupervised few-shot learning (U-FSL) aspires to bridge this gap, without relying on costly annotations. Inspired by the efficiency of contrastive representation learning, we propose a novel batch enhanced contrastive U-FSL pretraining methodology (coined as BECLR) to infuse instance- and class-level insights
within a contrastive framework. To enable the sampling of meaningful positives, we introduce an innovative dynamic clustered memory module (DyCE), which maintains highly-separable latent space partitions, through iterative equipartitioned updates. We also propose an effective, optimal transport (OT)-based feature alignment strategy (OpTA), to address sample bias in the U-FSL inference stage and further boost the end-to-end performance of BECLR in low-shot settings. Our extensive experimental evaluation corroborates the efficacy of our design choices
in BECLR, which sets a new state-of-the-art on the most widely adopted U-FSL benchmarks miniImageNet and tieredImageNet (offering up to 14% and 12% improvements, respectively), as well as on challenging cross-domain scenarios.

Training deep learning models for time-series prediction of a target population often requires a substantial amount of training data, which may not be readily available. This work addresses the challenge of leveraging multiple related sources of time series data in the same feature space to improve the prediction performance of a deep learning model for a target population. Specifically, we focus on a scenario where the target dataset, representing the desired target population, is underrepresented, while the source datasets consist of mismatched populations that are sufficiently representative for training a deep learning model. In this study, we explore state-of-the-art techniques, including transfer learning, ensemble learning, and domain adaptation to leverage source datasets towards a target population using real-world medical data. Additionally, we investigate the use of model performance-derived baselines as a heuristic to quantify the magnitude of the distribution mismatch between a source(s) and a target. Our results demonstrate that a set of well-defined baselines can effectively quantify the distribution mismatch and provide insights into the choice of leveraging technique for a given mismatch scenario. Furthermore, our results show that all state-of-the-art techniques can be employed to leverage related source datasets towards the target, though the performance of these techniques varies depending on the characteristics of the distribution mismatch. Eventually, we discuss the applicability of this research to new scenarios, along with avenues for future research.

Federated Learning is a machine learning paradigm for decentralized training over different clients. The training happens in rounds where each client learns a specific model which is then aggregated by a central server and passed back to the clients. Since the paradigm’s inception, many frameworks that provide Federated Learning tools and infrastructure have appeared. This leads to the question of ”How do different Federated Learning frameworks compare?”, which is the research question of this paper. The paper’s main contribution will be helping developers new to the Federated Learning field decide between NVidia Flare, OpenFL, and Flower, three popular federated learning frameworks.

Federated learning (FL) has emerged as a promis-ing approach for training machine learning models using geographically distributed data. This paper presents a comprehensive comparative study of var-ious machine learning models in the context of FL. The aim is to evaluate the efficacy of these models in different data distribution scenarios and provide
practical insights for practitioners in the field. The findings highlight the performance and limitations of linear and non-linear models on MNIST and Ki-nase datasets.

Neural networks are commonly initialized to keep the theoretical variance of the hidden pre-activations constant, in order to avoid the vanishing and exploding gradient problem. Though this condition is necessary to train very deep networks, numerous analyses showed that it is not sufficient. We explain this fact by analyzing the behavior of the empirical variance which is more meaningful in practice of data sets of finite size. We demonstrate its discrepancy with the theoretical variance which grows with depth. We study the output distribution of neural networks at initialization in terms of its kurtosis which we find to grow to infinity with increasing depth even if the theoretical variance stays constant. The result of this is that the empirical variance vanishes: its asymptotic distribution converges in probability to zero. Our analysis, which studies increased dependence of outputs, focuses on fully-connected ReLU networks with He initialization, but we hypothesize that many more random weight initialization methods suffer from either vanishing or exploding empirical variance. We support this hypothesis experimentally and demonstrate the failure of state-of-the-art random initialization methods in very deep regimes.