The rapid growth of the machine learning (ML) research field has significantly increased the volume of submissions to major conferences like NeurIPS, placing a substantial burden on peer reviewers. To preserve methodological rigour and transparency, conferences have introduced checklists; however, manually verifying compliance with these criteria remains highly timeconsuming. This thesis explores the feasibility of using a Large Language Model (LLM), specifically Google’s Gemini Flash Preview, to automate the assessment of the experimental statistical significance criterion on a dataset of 72 accepted NeurIPS 2024 papers. We evaluate the impact of iterative prompt design across three variations, shifting from rigid constraints to holistic, indicator-based guidelines. Our evaluation shows that while a stricter prompt results in high false-negative rates, an indicator-based prompt achieves an overall classification accuracy of 81% and a Cohen’s Kappa of 0.67 against human annotations. Qualitative analysis reveals that while the LLM successfully generates detailed justifications pointing out missing or present evidence, it occasionally struggles with broad context, such as mistaking baseline figures for primary experimental results. We conclude that current LLMs are effective as supportive reviewer tools to reduce manual checklist verification efforts, though they are not yet capable of replacing human judgment in contextual reasoning. ... Read more

As machine learning conferences such as NeurIPS expand rapidly, the manual classi-
fication and evaluation of responsible research checklists impose a significant burden on
reviewers. This study investigates the ability of Large Language Models (LLMs) to au-
tomatically classify research papers as empirical, theoretical, or hybrid, and to extract
checklist compliance data. Using a dataset of publicly available NeurIPS papers, we
designed an automated pipeline and evaluated its outputs against a human-annotated
ground truth. Our results demonstrate that the LLM achieves high accuracy in the
core classification task, reliably distinguishing the papers core methodology by iden-
tifying clear structural indicators like mathematical proofs and benchmark datasets.
Furthermore, the model excels at extracting objective checklist elements, performing
well on close-ended extraction tasks that rely on clear structural indicators. However,
performance noticeably decreased on structurally scattered or subjective criteria, such
as broader impacts and the declaration of AI usage. This drop highlights a limitation in
the model’s broader reading comprehension, as it struggles to merge contextual infor-
mation without explicit headers. Notably, this automated failure closely mirrors human
task ambiguity, as these exact subjective items also generated the lower inter-annotator
agreement among human annotators. Conclusively, while LLMs provide a highly con-
sistent baseline for classifying paper typologies and extracting explicit methodological
data, their reliance on structural cues indicates they should serve as assistive screening
tools rather than autonomous evaluators in academic peer review. ... Read more

Scientific peer review is an important part of the scientific process. However, the growing number of submissions has sparked interest in automated review tools. Recent work has shown that Large Language Models (LLMs) can generate reviews and evaluate author-provided checklists, yet it is unclear to what extent they can independently identify the scientific claims that are made in papers and perform structured reviews. This thesis investigates whether an LLM can automatically extract scientific claims from research papers in the machine learning field and then complete the NeurIPS Checklist without relying on author-written justifications. The evaluation focuses on claim extraction accuracy, preserving the semantic meaning of claims, and agreement between LLM-generated checklist annotations and human judgment. Gemini 3 Flash's claim extraction and checklist annotations are compared against human ground-truth annotations on NeurIPS 2024 papers. The results show that the model successfully identifies primary claims of papers, with a recall of 0.99 and precision of 0.75. Most errors are caused by over-segmentation or incorrect classification. For checklist annotation, the system achieves a mean accuracy of 0.85 and a mean Cohen's Kappa of 0.58 compared to human annotations. Agreement is strongest for objective checklist criteria. These findings indicate that LLMs can effectively support claim-based scientific review, but are not advanced enough to fully replace expert reviewers. ... Read more

The rapid growth of Machine Learning research has overwhelmed traditional peer-review systems, leading to concerns regarding reviewer fatigue and the consistency of scientific evaluation. While Large Language Models (LLMs) are being explored as potential assistants for quality assessment, their ability to objectively verify specific scientific criteria—such as those in the NeurIPS Paper Checklist—remains unproven. This checklist serves as a structured self-auditing framework that mandates authors to explicitly declare critical details, including potential negative societal impacts, exact hyperparameter tuning ranges, and clear definitions of theoretical assumptions or limitations. This study investigates the core question: “How well can an LLM extract the limitations described in scientific papers?” Using a manually annotated dataset of 78 papers, this research evaluates the accuracy of LLMs in extracting limitations stated by authors. Our findings reveal that while the LLM achieves perfect accuracy (100\%) in detecting the presence of dedicated limitation sections, its performance in textual extraction is more nuanced. For explicit limitations, the model demonstrates high recall (0.91) but moderate precision (0.71), frequently over-extracting context. Furthermore, when tasked with extracting implicit limitations from papers lacking dedicated sections, both recall (0.71) and precision (0.69) decline. Notably, we found that a major bottleneck in unstructured text is getting the LLM to look at the actual weakness instead of getting distracted by subsequent sentences talking about future work. By comparing LLM performance against a human-verified ground truth, this work provides a feasibility study for automating high-stakes research quality assessments and identifies current bottlenecks in LLM reasoning for scientific auditing. ... Read more

The recent surge in machine learning (ML) research has led to a record number of paper submissions, overwhelming the traditional peer-review process. Although conferences like NeurIPS have introduced reproducibility checklists to maintain scientific standards, manual verification of these claims is time-consuming and inconsistent. This study investigates the feasibility of using Large Language Models (LLMs) to automate the evaluation of paper reproducibility. By creating a ground-truth dataset through the manual annotation of NeurIPS papers, this study assesses the accuracy of LLMs in verifying author claims regarding code availability, hyperparameter transparency, and compute resources. The results compare LLM performance with manual labels to identify where automated tools succeed and where they fail to capture technical nuances. Ultimately, this research demonstrates that while LLMs can act as highly efficient administrative filters to streamline initial screening, they fail to reliably predict execution viability, highlighting the remaining boundaries of automated verification. ... Read more

In this thesis, we address the problem of learning mesh-specific, impulse-dependent fracture patterns in real time. Our approach is based on regressing a distance field over the mesh surface, encoding the proximity of each vertex to fracture lines, which is subsequently segmented into distinct pieces using graph-based methods such as watershed segmentation. The goal is to achieve real-time performance, which is something the current approach does not achieve for large meshes.

We evaluate different neural architectures, comparing a multilayer perceptron to DeltaConv, a graph convolutional model, and find that the MLP provides superior performance. In addition, we assess multiple segmentation strategies and identify watershed as the most effective, followed by hierarchical segmentation. We also find that the segmentation algorithms do not achieve real-time performance for large meshes.

These results highlight the potential of machine learning-based fracture simulations, but also indicate that distance field segmentation is not capable of real-time performance using our tested algorithms. This suggests that future work should focus on directly learning the labels rather than relying on distance fields as an intermediary representation in real-time scenarios.
... Read more

Accuracy‐driven recommender systems risk confining users to "filter‐bubbles'' of familiar content. Recent work on coVariance Neural Networks (VNNs) provides a scalable alternative to Principal Component Analysis (PCA) for modelling high-order correlations, but their impact on beyond-accuracy metrics (BAMs), such as Novelty and Diversity, remains unexplored.
We use the user–user covariance (or its inverse, the precision matrix) as a graph shift operator (GSO) and train SelectionGNN-based VNNs on the MovieLens-100K dataset.
Two training regimes are evaluated: (i) RMSE-only (No-BAM-SVNN) and (ii) a compound loss that also includes novelty and diversity terms (BAM-SVNN).
For each regime we sweep six graph configurations: covariance/precision crossed with {dense, hard-threshold, soft-threshold} sparsification, under five random seeds, yielding 30 runs per regime.
Baseline comparisons include PCA, a naive mean–std model, and a random predictor.

The best SVNN configuration increases recommendation Novelty by 2.8 percentage points and matches PCA’s Diversity while incurring only a 0.03 RMSE penalty.
Hard-thresholded precision graphs provide the lowest SVNN RMSE (0.952), whereas dense covariance graphs maximise diversity (0.868).
Integrating novelty/diversity directly into the loss offers no additional benefit yet multiplies runtime by x33.
One-way ANOVA indicates that model family explains 97.6% of RMSE variance (\(\eta^2=0.976\)) and 77.8% of novelty variance.

This work is the first to benchmark (sparsified) VNNs on beyond-accuracy metrics, demonstrating a favourable accuracy–novelty trade-off and clarifying when sparsification and BAM-weighted training pay off.
All code, data splits and statistical notebooks are released for full reproducibility. ... Read more

Recommender systems help users navigate vast catalogs of content through recommendations, of which rating prediction remains an important task. Traditional methods such as collaborative filtering often struggle to model higher-order relationships between users and items, as well as suffer from the cold start problem when the number of users and items is still low. Graph Neural Networks (GNNs) have shown promise in this area, although they are often limited by their focus on local graph structures. This study explores the application of Covariance Neural Networks (VNNs) for rating prediction, leveraging covariance matrices to leverage global statistical dependencies and model higher-order relationships. Using the MovieLens-100k dataset, we evaluate the performance of VNNs against baselines and other models, using RMSE as the metric of evaluation. Our results demonstrate that VNNs outperform simple matrix completion techniques, but are limited by their susceptibility to oversmoothing. This work highlights the potential of VNNs for recommender systems while underscoring the need for careful architectural design to balance performance and stability. ... Read more

This research investigates the application of Graph Neural Networks (GNNs) for rating prediction in recommender systems, utilizing precision matrices as graph filters. The focus is on movie recommendation, where graph-based structures are especially relevant due to the importance of user and item relationships. A leave-one-out masking strategy is employed during training to ensure the model learns from all available training data. The proposed model achieves a test root mean squared error (RMSE) of approximately 0.95 on the MovieLens-100k dataset, performing reasonably well compared to existing graph-based and matrix factorization methods. While the model accurately predicts average ratings, it tends to overestimate lower ratings. These results demonstrate the potential of precision-based graph filters in GNNs but also reveal significant room for improvement before reaching state-of-the-art performance. Future work may include output calibration and sparsification of the precision matrix to enhance both efficiency and predictive accuracy. ... Read more

Graph Neural Networks (GNNs) are an effective architecture for implementing collaborative filtering-based recommender systems. This paper evaluates the performance and computational complexity of precision matrix-based VNNs as a collaborative filter on the MovieLens-100K dataset. Results show the estimated precision matrix contains a high amount of noise when calculated from sparse data, which impacts the performance of the model. After sparsifying the precision matrix, the performance and computational complexity improved significantly. ... Read more

High-resolution image analysis using deep Convolutional Neural Networks (CNNs) faces significant memory constraints due to the quadratic growth of intermediate feature maps with input resolution. This paper investigates whether Implicit Neural Representations (INRs), specifically SIRENs, can effectively represent CNN feature maps to reduce memory footprint during training. We address the unique challenge that CNN feature maps are not static signals but evolve continuously as network weights are updated through gradient-based optimization. Through three experiments on a modified All-CNN architecture trained on MNIST, we validate that: (1) SIRENs can fit static feature maps from frozen CNNs with high fidelity (PSNR > 30 dB) regardless of weight initialization; (2) SIRENs can track evolving feature maps during training, though with reduced reconstruction quality compared to static targets; and (3) SIREN-assisted feedforward—where SIRENs predict missing activations in receptive fields—enables classification accuracy (20.97%) above random guessing (10%) but substantially below standard training (95%). While results demonstrate the feasibility of using SIRENs to represent dynamic feature maps, significant challenges remain in maintaining reconstruction fidelity when SIRENs are integrated into the training loop. This proof-of-concept study provides empirical insights into bridging continuous implicit representations with discrete deep learning pipelines and highlights promising directions for future research in memory-efficient high-resolution image analysis. ... Read more

Alzheimer’s disease (AD) is a neurodegenerative disorder prevalent in older adults, leading to loss in memory, cognitive, and executive function. A characteristic feature of AD is the accumulation of amyloid-beta (Aβ) plaques, which are extracellular deposits of Aβ protein primarily found in grey matter. These Aβ deposits can appear in different forms. The six primary Aβ deposits covered in this work are: diffuse plaques, cored plaques, compact plaques, coarse grained plaques, cerebral amyloid angiopathy (CAA), and subpial deposits. It is speculated that some of the plaques types may be more harmful than others. Given that AD primarily affects an older population, the question arises of how individuals with AD differentiate from cognitively healthy centenarians (people over 100 years old). Consequently, this work attempts to classify and analyse different Aβ types present in donated brain tissue of cognitively healthy centenarians who escaped dementia and individuals diagnosed with AD. The main goal is to identify differences between these two cohorts. However, a challenge in this process is that the brain tissues contain many Aβ plaques, making manual identification difficult in terms of time and labor. To address this, a fine-tuned ResNet50 Aβ plaque classifier was developed in this research that was integrated into an Aβ detection pipeline capable of localising plaques. The model was initially pre-trained on the ImageNet dataset through contrastive learning, and subsequently fine-tuned using few-shot learning with a small number of annotated samples (315). The annotations included the six primary Aβ types whose structure are known and well-defined, and three other anomaly Aβ types that
served to filter out irregular plaques in the unlabeled Aβ data. After performing 5-fold cross-validation, the fine-tuned models demonstrated an average accuracy of 85.71% and precision of 89.47% on the primary types. The final classifier used on the unlabeled Aβ dataset was an ensemble model that incorporated majority voting, combining the predictions of the five models trained during cross-validation. Aβ loads were calculated for each primary Aβ type based on the classifier’s predictions. It was observed that across all considered primary Aβ types, the centenarians’ Aβ loads were statistically significantly lower compared to the AD cohort. The lower Aβ load in centenarians also held true across the frontal, temporal, parietal, and occipital cerebral regions for each primary Aβ type. To partially validate the
model’s performance, correlations were computed between the predicted Aβ loads of the primary Aβ types and neuropathological assessments collected from 75 centenarians. These assessments are common in related works and serve as a benchmark for the ensemble model. They include the Thal Aβ phase, which categorizes the distribution of general Aβ in the brain; the Thal CAA stage, which measures the severity of CAA; and CERAD NP scores, which evaluates the spread of neuritic plaques in the brain, which are a subset of cored plaques. Consequently, statistically significant positive correlations were revealed between: the Thal Aβ phase and the Aβ load of all primary types (r ranging 0.59-0.73); the Thal CAA stage and Aβ load of predicted CAA deposits (r = 0.66); and the CERAD NP scores and Aβ load of cored plaques (r = 0.68). Since the correlated types coincide with what the benchmark staging schemes measure, the model’s predictions seems to align with existing literature. ... Read more

Graph Neural Networks are widely used as useful tools to investigate graphs because they can learn from the topological structure of graphs. In practical applications, the graph’s structure can change over time, have errors or be subject to adversarial attacks. These perturbations negatively impact the accuracy of the neural network. The theoretical stability of graph neural networks has been analysed already and in this paper, the stability of graph neural networks is investigated experimentally. The performance of different perturbation strategies is compared to see how different perturbations impact stability. ... Read more

GNNs are a powerful tool for learning tasks on data with a graph structure. However, the topology of the graph in which GNNs are trained is often subject to change due to random, external perturbations. This research investigates the relationship between 5 topological properties of graphs (assortativity, density, edge connectivity, closeness centrality, diameter) and how stable GNNs trained on graphs with different topological properties are against different perturbations. The analysis is conducted by first synthetically generating graphs with different topological properties and training a GNN using the generated graphs. The synthetic graphs are then perturbed, and the relative change in the GNNs' output is measured. These results are further supported by conducting the same process on three popular GNN datasets: Cora, CiteSeer and PubMed citations. Finally, relationships between the graph properties under investigation and GNN stability are inferred using the results obtained from both synthetic and real-world datasets. ... Read more

Graph Neural Networks (GNNs) have emerged as a powerful tool for learning from relational data. The real-world graphs such models are trained on are susceptible to changes in their topology. A growing body of work in the field of GNNs' stability to topology perturbations is trying to characterise how models respond to those changes, providing valuable insights that have enhanced the robustness of GNNs to adversarial attacks. The past work in this field, however, has only approached stability analysis using spectral graph theory, which is not applicable to all kinds of GNN models. In this paper, we aim to extend the past work in GNN stability by proposing an algorithm for analysing the stability of GNNs with model-agnostic GNN explainability tools instead of the mathematical framework of spectral graph theory. We demonstrate that the outputs of explainability tools can encode useful insights into the stability of GNNs and present a case study on using those insights to analyse node removal, edge removal, and edge weight perturbations. ... Read more

Graph Neural Network holds significant impor- tance in various applications. Pioneering research has demonstrated state-of-the-art performance in practical applications such as Fraud Detection, Recommender Systems, or Traffic Forecasting by utilizing various Graph Neural Networks (GNNs) architectures. For these applications, one of the most important properties that needs to hold is the stability of GNN under stochastic perturbation as real-life networks undergo changes in topology on a frequent basis. However, it remains unclear how different architectures preserve this property under different perturbations. In this research, we aim to shed light on if this stability property undergoes drastic changes in the graph underlying topology, and if it affects the overall performance of the GNN in Traffic Forecasting problems. We demonstrate that the architectures differ in the stability property measured by different metrics, while some archi- tectures retains their state-of-the-art performance, providing useful insight on the analysis of stability property on different graph neural network archi- tectures in Traffic Forecasting problem. ... Read more

The detection of changes in an area over time using remotely sensed data such as images is referred to as change detection. It has a large range of applications. For example, changes in buildings can analysed for urban planning. Many conventional image processing and machine learning-based algorithms have been developed for the purpose of change detection. Conventional non-classification algorithms have advantages in their reduced computational cost. Remotely sensed images vary in their spatial resolution, which is the area a pixel covers on the Earth surface. This work aims to explore how the spatial resolution impacts conventional non-classification pixel-based techniques in the urban change detection context, to provide insight into their performance with regards to detecting urban-related change over different resolutions. A systematic experiment is conducted by considering the LEVIR-CD and OSCD test sets in their initial as well as multiple downsampled resolutions. The change detection algorithms Change Vector Analysis (CVA) and Iteratively Reweighted Multivariate Alteration Detection (IR-MAD) are applied to the data individually. For creating binary change labels on a pixel-level, the Otsu algorithm is applied. A set of performance metrics is calculated, and trends in the metrics values over the resolutions are analysed. The data shows some trends towards improved metric values for lower spatial resolutions. The degree of the trends varies and is dependent on the algorithm and dataset. Overall, further research is necessary to consider influencing factors such as the amount of pixels in images, to refine the processing steps, and to broaden the scope of the experiment. ... Read more

As global urbanization reaches an all-time high, effective urban management becomes a crucial factor for efficient development. Enhanced monitoring of these transformations leads to more informed decision-making by policymakers, emphasizing the importance of tracking these changes. One method for monitoring is Change Detection (CD), which involves comparing two satellite images captured at different times to detect changes over a period of time. CD involves numerous difficulties, such as data collection, varying weather conditions, limited availability of datasets, noise, illumination differences, and discrepancies in the equipment used for image capture. Convolutional Neural Networks (CNNs) can address these issues by delivering more effective models with better performance than non-deep learning models. However, the rise of Transformers has led researchers to develop networks based on Transformer architecture, yielding more promising results than CNNs when more data is available. This paper conducts an analysis of two existing Transformer-based models, emphasizing the challenges of handling CD with artificially small datasets. Using smaller datasets reduces the requirements for remote sensing capabilities of satellites, simulating the limitations encountered during data collection and processing. The models under examination are the Bitemporal Image Transformer (BIT) and the Visual change Transformer (VcT). ... Read more

Change detection with remote sensing data highlights se- mantic differences in an area between two or more time intervals. It involves the comparison of aerial photographs of the same location taken some time apart. This faci itates mass scale analysis of urban and rural data over time, including population trends, city expansion trends and illegal building detection. State-of-the-art methods for the task are predominantly deep learning networks, following an encoder-decoder architecture. These architectures all share the trait of having a ”fusion” point - a location in the network where inputs transition from being processed independently to becoming correlated. F sions can be classified in three categories: early, middle and late, depending on how deep within the network they occur. This study aims to show how changing the fusion impacts the size, spread and number of changes detected. It is motivated by how the receptive field of feature maps in convolutional neural networks expands in deeper layers, extracting features with different complexities. For this, four fusion architectures on three different datasets are compared: LEVIR-CD, HiUCD and a new, fully-controled dataset, CSCD. In terms of test accuracy and the changes’ size and spread, results are inconclusive. Which fusion achieves the highest performance varies per dataset. Possible reasons why include the complexity of remote sensing data and general differences between areas, but this is a subject of further study. The only conclusive category is the number of changes detected. On aver- age, all architectures overestimate the number of changes in a scene. When the accuracy of architectures is com- parable, however, early fusion overestimates the number of objects changed the most, while middle and late fusion give more realistic estimates. The case study has room for refinement in problem isolation, more data and extending the problem towards more architectures, but is a promising step towards understanding fusion. ... Read more

Urban change detection involves identifying and analyzing alterations in urban landscapes over time. This process is crucial for urban planning, environmental monitoring, and disaster management, as it provides insights into urban growth, land use changes, and human impact on the environment. This study focuses on Recurrent Neural Networks (RNNs) due to their ability to capture temporal dependencies, making them suitable for analyzing changes over time. In this research, an RNN model, specifically the SiamCRNN, was trained and evaluated on multiple datasets representing different urban scenarios. The model performed well in detecting urban changes, especially in datasets with higher spatial resolutions, but faced challenges with datasets characterized by high spectral range and complex urban structures. These findings underscore the importance of spatial resolution in influencing RNN model effectiveness for urban change detection tasks. ... Read more