A. van Deursen
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
92 records found
1
Financial crime detection requires identifying rare illicit activity within large transaction networks, where suspicious behaviour can depend on both relationships between accounts and patterns in their transaction histories. Existing graph-based methods capture structural information, yet typically represent temporal behaviour only through transaction attributes or shared account representations. This thesis investigates whether representations from pretrained time-series foundation models (TSFMs) can provide richer context for graph-based financial crime detection. We introduce Temporal-Context Edge Enrichment (TCEE), which uses Chronos-2, a pretrained TSFM, to encode account transaction histories and enables each transaction to condition the retrieval of account-history context on its own features. Unlike approaches that assign all transactions of an account the same representation, TCEE constructs a transaction-specific representation conditioned on the transaction itself. These edge representations can be classified directly or given to a graph neural network (GNN) for additional structural information. Experiments on six AMLWorld anti-money laundering datasets show that TCEE without a GNN backbone improves PR-AUC over the strongest evaluated baseline on five datasets by 10.8% on average, while reaching its best validation PR-AUC 6.0 times faster end-to-end on average across all six datasets, including the one-time cost of Chronos-2 encoding. Adding a GNN backbone improves PR-AUC by a further 4.8% over TCEE without the backbone on the four datasets trained to convergence, and 2.4% on average across all six. On a real Ethereum phishing graph, Chronos-PNA improves PR-AUC by 14.5% over the strongest evaluated baseline, while full TCEE-PNA improves it by 17.8%, showing that Chronos-2 representations can also transfer beyond anti-money laundering when combined with graph learning.
...
Financial crime detection requires identifying rare illicit activity within large transaction networks, where suspicious behaviour can depend on both relationships between accounts and patterns in their transaction histories. Existing graph-based methods capture structural information, yet typically represent temporal behaviour only through transaction attributes or shared account representations. This thesis investigates whether representations from pretrained time-series foundation models (TSFMs) can provide richer context for graph-based financial crime detection. We introduce Temporal-Context Edge Enrichment (TCEE), which uses Chronos-2, a pretrained TSFM, to encode account transaction histories and enables each transaction to condition the retrieval of account-history context on its own features. Unlike approaches that assign all transactions of an account the same representation, TCEE constructs a transaction-specific representation conditioned on the transaction itself. These edge representations can be classified directly or given to a graph neural network (GNN) for additional structural information. Experiments on six AMLWorld anti-money laundering datasets show that TCEE without a GNN backbone improves PR-AUC over the strongest evaluated baseline on five datasets by 10.8% on average, while reaching its best validation PR-AUC 6.0 times faster end-to-end on average across all six datasets, including the one-time cost of Chronos-2 encoding. Adding a GNN backbone improves PR-AUC by a further 4.8% over TCEE without the backbone on the four datasets trained to convergence, and 2.4% on average across all six. On a real Ethereum phishing graph, Chronos-PNA improves PR-AUC by 14.5% over the strongest evaluated baseline, while full TCEE-PNA improves it by 17.8%, showing that Chronos-2 representations can also transfer beyond anti-money laundering when combined with graph learning.
Dependently typed languages and proof assistants such as Agda and Rocq improve the reliability of mathematical proofs and programs by representing logical propositions as types and proofs as programs inhabiting those types. Through the Curry--Howard correspondence, typechecking becomes proof checking, allowing correctness guarantees to be established by construction.
However, these guarantees ultimately rely on the correctness of the language implementation itself. Components such as typecheckers, conversion checkers, and termination checkers are typically implemented as algorithms whose correctness must be trusted independently from the theory they are intended to enforce.
Agda Core is a core language for Agda implemented in Agda itself, where language judgements are represented directly as dependent types and checking procedures become programs constructing evidence of those judgements. This thesis investigates how the same methodology can be applied to termination checking.
Termination checking is a fundamental component of dependently typed languages, since unrestricted recursion may compromise normalization and logical consistency. We explore how termination criteria can be represented as formal specifications together with executable proof-search procedures producing explicit certificates that the criteria hold.
We first study the guard condition, a structural recursion criterion based on descending recursive arguments, and implement a certified checker producing explicit evidence that the criterion holds. We then investigate the size change principle, a stronger termination criterion capable of handling a wider class of recursive and mutually recursive definitions. We formalize the corresponding rules in the same framework and discuss the challenges involved in constructing a fully executable checker for them.
More generally, this work explores the separation between declarative specifications of termination criteria and the algorithms used to search for certificates satisfying them. By expressing termination arguments directly in the type theory of the implementation language, the resulting infrastructure becomes simultaneously executable, inspectable, and partially verified.
...
However, these guarantees ultimately rely on the correctness of the language implementation itself. Components such as typecheckers, conversion checkers, and termination checkers are typically implemented as algorithms whose correctness must be trusted independently from the theory they are intended to enforce.
Agda Core is a core language for Agda implemented in Agda itself, where language judgements are represented directly as dependent types and checking procedures become programs constructing evidence of those judgements. This thesis investigates how the same methodology can be applied to termination checking.
Termination checking is a fundamental component of dependently typed languages, since unrestricted recursion may compromise normalization and logical consistency. We explore how termination criteria can be represented as formal specifications together with executable proof-search procedures producing explicit certificates that the criteria hold.
We first study the guard condition, a structural recursion criterion based on descending recursive arguments, and implement a certified checker producing explicit evidence that the criterion holds. We then investigate the size change principle, a stronger termination criterion capable of handling a wider class of recursive and mutually recursive definitions. We formalize the corresponding rules in the same framework and discuss the challenges involved in constructing a fully executable checker for them.
More generally, this work explores the separation between declarative specifications of termination criteria and the algorithms used to search for certificates satisfying them. By expressing termination arguments directly in the type theory of the implementation language, the resulting infrastructure becomes simultaneously executable, inspectable, and partially verified.
...
Dependently typed languages and proof assistants such as Agda and Rocq improve the reliability of mathematical proofs and programs by representing logical propositions as types and proofs as programs inhabiting those types. Through the Curry--Howard correspondence, typechecking becomes proof checking, allowing correctness guarantees to be established by construction.
However, these guarantees ultimately rely on the correctness of the language implementation itself. Components such as typecheckers, conversion checkers, and termination checkers are typically implemented as algorithms whose correctness must be trusted independently from the theory they are intended to enforce.
Agda Core is a core language for Agda implemented in Agda itself, where language judgements are represented directly as dependent types and checking procedures become programs constructing evidence of those judgements. This thesis investigates how the same methodology can be applied to termination checking.
Termination checking is a fundamental component of dependently typed languages, since unrestricted recursion may compromise normalization and logical consistency. We explore how termination criteria can be represented as formal specifications together with executable proof-search procedures producing explicit certificates that the criteria hold.
We first study the guard condition, a structural recursion criterion based on descending recursive arguments, and implement a certified checker producing explicit evidence that the criterion holds. We then investigate the size change principle, a stronger termination criterion capable of handling a wider class of recursive and mutually recursive definitions. We formalize the corresponding rules in the same framework and discuss the challenges involved in constructing a fully executable checker for them.
More generally, this work explores the separation between declarative specifications of termination criteria and the algorithms used to search for certificates satisfying them. By expressing termination arguments directly in the type theory of the implementation language, the resulting infrastructure becomes simultaneously executable, inspectable, and partially verified.
However, these guarantees ultimately rely on the correctness of the language implementation itself. Components such as typecheckers, conversion checkers, and termination checkers are typically implemented as algorithms whose correctness must be trusted independently from the theory they are intended to enforce.
Agda Core is a core language for Agda implemented in Agda itself, where language judgements are represented directly as dependent types and checking procedures become programs constructing evidence of those judgements. This thesis investigates how the same methodology can be applied to termination checking.
Termination checking is a fundamental component of dependently typed languages, since unrestricted recursion may compromise normalization and logical consistency. We explore how termination criteria can be represented as formal specifications together with executable proof-search procedures producing explicit certificates that the criteria hold.
We first study the guard condition, a structural recursion criterion based on descending recursive arguments, and implement a certified checker producing explicit evidence that the criterion holds. We then investigate the size change principle, a stronger termination criterion capable of handling a wider class of recursive and mutually recursive definitions. We formalize the corresponding rules in the same framework and discuss the challenges involved in constructing a fully executable checker for them.
More generally, this work explores the separation between declarative specifications of termination criteria and the algorithms used to search for certificates satisfying them. By expressing termination arguments directly in the type theory of the implementation language, the resulting infrastructure becomes simultaneously executable, inspectable, and partially verified.
LLM-driven Malware Analysis across Code Representations
A Study on the Impact of Code Representation on the Performance of LLM-driven Malware Classification
In this study, we determine the extent to which code representation affects the accuracy of LLM-driven malware analysis. Our results show that LLMs perform significantly better at detecting malware in high-level source code than in binary code. We conduct experiments on samples from the SBAN dataset. In the process, we also evaluate the validity of the SBAN dataset as a benchmark for malware classification, helping direct future efforts toward improving dataset quality.
...
In this study, we determine the extent to which code representation affects the accuracy of LLM-driven malware analysis. Our results show that LLMs perform significantly better at detecting malware in high-level source code than in binary code. We conduct experiments on samples from the SBAN dataset. In the process, we also evaluate the validity of the SBAN dataset as a benchmark for malware classification, helping direct future efforts toward improving dataset quality.
Can LLMs Consistently Describe Programs Across Source Code, Assembly, and Binary Representations?
Evaluating the Quality of Generated High-Level Descriptions of Benign and Malware Programs
Large language models (LLMs) are increasingly used to summarize and reason about software artifacts. This is especially true in cybersecurity, where analysts must often interpret low-level code such as assembly or binary. If LLMs describe the same program differently depending on its representation, analysts may therefore receive inconsistent or incomplete explanations. This paper evaluates whether an LLM generates descriptions that are both consistent across representations of the same program and aligned with human reference descriptions. Using a balanced subset of SBAN, a dataset which provides aligned high-level source code, disassembled assembly, and a raw-hexadecimal binary representation of the same programs together with a natural-language reference, we generate high-level descriptions for every representation with Qwen3.5-2B using a fixed prompt and low-temperature stochastic decoding, repeated over five runs for 75000 descriptions in total. To prevent context-level reference leakage and support reproducibility, each description is generated independently, without conversation history, the reference description, dataset labels, or the other representations. We measure cross-representation consistency and reference alignment with complementary metrics: sentence-transformer cosine similarity and ROUGE-L over the full dataset, BERTScore against the references, and Prometheus, an independent LLM judge, on a fixed 600-sample subset. Source-code descriptions align best with the references and assembly-source descriptions are the most consistent, while binary-source is the least consistent. A Friedman test confirms a statistically significant representation effect on reference-based quality. The absolute differences are small, however, and cross-representation consistency is only moderate across all metrics. These results indicate that representation choice measurably affects both the quality and consistency of LLM-generated descriptions, likely because each representation exposes a different level of semantic information.
...
Large language models (LLMs) are increasingly used to summarize and reason about software artifacts. This is especially true in cybersecurity, where analysts must often interpret low-level code such as assembly or binary. If LLMs describe the same program differently depending on its representation, analysts may therefore receive inconsistent or incomplete explanations. This paper evaluates whether an LLM generates descriptions that are both consistent across representations of the same program and aligned with human reference descriptions. Using a balanced subset of SBAN, a dataset which provides aligned high-level source code, disassembled assembly, and a raw-hexadecimal binary representation of the same programs together with a natural-language reference, we generate high-level descriptions for every representation with Qwen3.5-2B using a fixed prompt and low-temperature stochastic decoding, repeated over five runs for 75000 descriptions in total. To prevent context-level reference leakage and support reproducibility, each description is generated independently, without conversation history, the reference description, dataset labels, or the other representations. We measure cross-representation consistency and reference alignment with complementary metrics: sentence-transformer cosine similarity and ROUGE-L over the full dataset, BERTScore against the references, and Prometheus, an independent LLM judge, on a fixed 600-sample subset. Source-code descriptions align best with the references and assembly-source descriptions are the most consistent, while binary-source is the least consistent. A Friedman test confirms a statistically significant representation effect on reference-based quality. The absolute differences are small, however, and cross-representation consistency is only moderate across all metrics. These results indicate that representation choice measurably affects both the quality and consistency of LLM-generated descriptions, likely because each representation exposes a different level of semantic information.
Malware-Domain Continued Pre-Training for Binary Malware Classification
A Leakage-Aware Study of Code Models on the SBAN Corpus
Continued pre-training can adapt language models to a domain, but for malware classification it is un- clear whether gains come from malware-specific information or from additional training on code. We study this question on a leakage-controlled binary benchmark derived from the SBAN cor- pus, using strict BENIGN/MALWARE labels, exact duplicate removal, and matched compar- isons between TF-IDF baselines, CodeBERT, and Qwen2.5-Coder variants. Across the tested model sizes and pre-training data budgets, continued pre- training changes model behaviour but does not pro- duce a reliable downstream classification improve- ment. Even in the best malware-related setting, the margin over an equally trained general-code control is very small. Under these constraints, the overall trend is that malware-related continued pre-training does not improve binary classification in a reliable way.
...
Continued pre-training can adapt language models to a domain, but for malware classification it is un- clear whether gains come from malware-specific information or from additional training on code. We study this question on a leakage-controlled binary benchmark derived from the SBAN cor- pus, using strict BENIGN/MALWARE labels, exact duplicate removal, and matched compar- isons between TF-IDF baselines, CodeBERT, and Qwen2.5-Coder variants. Across the tested model sizes and pre-training data budgets, continued pre- training changes model behaviour but does not pro- duce a reliable downstream classification improve- ment. Even in the best malware-related setting, the margin over an equally trained general-code control is very small. Under these constraints, the overall trend is that malware-related continued pre-training does not improve binary classification in a reliable way.
Optimized Concept Learning in BEN
Better identifying when transformations apply in a program synthesis agent
The Abstraction and Reasoning Corpus (ARC) challenges AI systems to induce general rules from as few as two to four examples. BEN is a program synthesis agent that tackles ARC through a Divide, Align, and Conquer pipeline inspired by analogical reasoning, but its concept learning phase, which synthesizes Boolean rules governing when transformations apply, must frequently choose between candidate rules of equal structural complexity. Because ARC's extreme data scarcity makes such ties highly prevalent, the system's coarse notion of complexity provides insufficient nuance to distinguish between candidates, leaving it vulnerable to overfitting.
We present three optimizations to BEN's concept learner, built on a reimplementation of BEN in Julia. These optimization provide better nuance in complexity at different stages of the pipeline, helping the system more accurately mimic human analogical reasoning. A \emph{Rule Generality} heuristic uses cross-transformation frequency analysis to discount broadly applicable features. A \emph{DNF Model Score} leverages cumulative solver objective values as a semantic tie-breaker between structurally equivalent rules. A \emph{Simple Minimal Transformation Coverage} heuristic extends the greedy set-cover procedure, with secondary criteria based on correspondence ambiguity and transformation complexity. An ablation study on 400 ARC-AGI-1 tasks shows these optimizations increase accuracy from 32 to 36 solved tasks with no regressions and negligible computational overhead. Analysis of rule complexity reveals that generated DNFs consistently require only a single clause, suggesting that the current performance bottleneck lies in the upstream pipeline rather than in concept complexity itself. ...
We present three optimizations to BEN's concept learner, built on a reimplementation of BEN in Julia. These optimization provide better nuance in complexity at different stages of the pipeline, helping the system more accurately mimic human analogical reasoning. A \emph{Rule Generality} heuristic uses cross-transformation frequency analysis to discount broadly applicable features. A \emph{DNF Model Score} leverages cumulative solver objective values as a semantic tie-breaker between structurally equivalent rules. A \emph{Simple Minimal Transformation Coverage} heuristic extends the greedy set-cover procedure, with secondary criteria based on correspondence ambiguity and transformation complexity. An ablation study on 400 ARC-AGI-1 tasks shows these optimizations increase accuracy from 32 to 36 solved tasks with no regressions and negligible computational overhead. Analysis of rule complexity reveals that generated DNFs consistently require only a single clause, suggesting that the current performance bottleneck lies in the upstream pipeline rather than in concept complexity itself. ...
The Abstraction and Reasoning Corpus (ARC) challenges AI systems to induce general rules from as few as two to four examples. BEN is a program synthesis agent that tackles ARC through a Divide, Align, and Conquer pipeline inspired by analogical reasoning, but its concept learning phase, which synthesizes Boolean rules governing when transformations apply, must frequently choose between candidate rules of equal structural complexity. Because ARC's extreme data scarcity makes such ties highly prevalent, the system's coarse notion of complexity provides insufficient nuance to distinguish between candidates, leaving it vulnerable to overfitting.
We present three optimizations to BEN's concept learner, built on a reimplementation of BEN in Julia. These optimization provide better nuance in complexity at different stages of the pipeline, helping the system more accurately mimic human analogical reasoning. A \emph{Rule Generality} heuristic uses cross-transformation frequency analysis to discount broadly applicable features. A \emph{DNF Model Score} leverages cumulative solver objective values as a semantic tie-breaker between structurally equivalent rules. A \emph{Simple Minimal Transformation Coverage} heuristic extends the greedy set-cover procedure, with secondary criteria based on correspondence ambiguity and transformation complexity. An ablation study on 400 ARC-AGI-1 tasks shows these optimizations increase accuracy from 32 to 36 solved tasks with no regressions and negligible computational overhead. Analysis of rule complexity reveals that generated DNFs consistently require only a single clause, suggesting that the current performance bottleneck lies in the upstream pipeline rather than in concept complexity itself.
We present three optimizations to BEN's concept learner, built on a reimplementation of BEN in Julia. These optimization provide better nuance in complexity at different stages of the pipeline, helping the system more accurately mimic human analogical reasoning. A \emph{Rule Generality} heuristic uses cross-transformation frequency analysis to discount broadly applicable features. A \emph{DNF Model Score} leverages cumulative solver objective values as a semantic tie-breaker between structurally equivalent rules. A \emph{Simple Minimal Transformation Coverage} heuristic extends the greedy set-cover procedure, with secondary criteria based on correspondence ambiguity and transformation complexity. An ablation study on 400 ARC-AGI-1 tasks shows these optimizations increase accuracy from 32 to 36 solved tasks with no regressions and negligible computational overhead. Analysis of rule complexity reveals that generated DNFs consistently require only a single clause, suggesting that the current performance bottleneck lies in the upstream pipeline rather than in concept complexity itself.
Seeing the Gaps: Improving Object Segmentation for Abstract Visual Reasoning in Julia
Grammar Extensions and Structural Ranking for the BEN Agent on the ARC Benchmark
The Abstraction and Reasoning Corpus (ARC) is a benchmark designed to measure general-purpose skill acquisition, requiring solvers to infer transformation rules from very few examples. Program synthesis approaches such as the Divide, Align and Conquer (DA&C) framework have shown promise, but their segmentation stage, which decomposes input grids into objects, remains a bottleneck in both computational cost and task coverage. This work presents a reimplementation of the BEN agent in Julia, integrated with the Herb.jl program synthesis ecosystem, alongside two targeted improvements to the segmentation module. First, we extend the segmentation grammar with a proximity;N mode that groups same-color pixels within Chebyshev distance N [1], enabling correct decomposition of objects with small internal gaps. Second, we replace the original full-pipeline mode selection with a lightweight structural ranking that scores all candidate modes on their segmentation output alone, using all training examples rather than only the first. Evaluated on the 400-task ARC training set with a 140-second budget, the Julia reimplementation solves 46 tasks, of which 5 are solved via proximity modes absent from the original grammar and are therefore likely attributable to the grammar extension. Analysis of the ranking reveals that the top-ranked mode solves 59% of solvable tasks. A deliberate fallback mechanism compensates for the heuristic's imperfection by guaranteeing that a reliable base mode is always attempted second. Grammar extensions account for some improvement (5 out of 13 tasks likely solved exclusively by Julia).
...
The Abstraction and Reasoning Corpus (ARC) is a benchmark designed to measure general-purpose skill acquisition, requiring solvers to infer transformation rules from very few examples. Program synthesis approaches such as the Divide, Align and Conquer (DA&C) framework have shown promise, but their segmentation stage, which decomposes input grids into objects, remains a bottleneck in both computational cost and task coverage. This work presents a reimplementation of the BEN agent in Julia, integrated with the Herb.jl program synthesis ecosystem, alongside two targeted improvements to the segmentation module. First, we extend the segmentation grammar with a proximity;N mode that groups same-color pixels within Chebyshev distance N [1], enabling correct decomposition of objects with small internal gaps. Second, we replace the original full-pipeline mode selection with a lightweight structural ranking that scores all candidate modes on their segmentation output alone, using all training examples rather than only the first. Evaluated on the 400-task ARC training set with a 140-second budget, the Julia reimplementation solves 46 tasks, of which 5 are solved via proximity modes absent from the original grammar and are therefore likely attributable to the grammar extension. Analysis of the ranking reveals that the top-ranked mode solves 59% of solvable tasks. A deliberate fallback mechanism compensates for the heuristic's imperfection by guaranteeing that a reliable base mode is always attempted second. Grammar extensions account for some improvement (5 out of 13 tasks likely solved exclusively by Julia).
Chain-of-Thought LLM-Based Code Translation
Using Chain-of-Thought to Improve LLM-Based Translation of C++ to Java
Large language models (LLMs) have recently gained significant traction for their ability to assist with day-to-day tasks, especially coding-related tasks. At the same time, cybersecurity threats are becoming increasingly complex, sometimes requiring thorough analysis by multiple experts in the field. Therefore, it is valuable to assess the ability of LLMs to aid in malware inspection.
This research focuses on the use of LLMs as a code translation tool for beginner malware researchers. Novices can use such a tool to translate malware source code from C++ to Java, helping them understand its functionality by making the code more readable. Two zero-shot prompting frameworks are presented and evaluated for their effectiveness, achieving syntactically correct outputs at rates between 62.2% and 63.9%. Of these outputs, between 39.0% and 42.9% produce functionally preserved translations. ...
This research focuses on the use of LLMs as a code translation tool for beginner malware researchers. Novices can use such a tool to translate malware source code from C++ to Java, helping them understand its functionality by making the code more readable. Two zero-shot prompting frameworks are presented and evaluated for their effectiveness, achieving syntactically correct outputs at rates between 62.2% and 63.9%. Of these outputs, between 39.0% and 42.9% produce functionally preserved translations. ...
Large language models (LLMs) have recently gained significant traction for their ability to assist with day-to-day tasks, especially coding-related tasks. At the same time, cybersecurity threats are becoming increasingly complex, sometimes requiring thorough analysis by multiple experts in the field. Therefore, it is valuable to assess the ability of LLMs to aid in malware inspection.
This research focuses on the use of LLMs as a code translation tool for beginner malware researchers. Novices can use such a tool to translate malware source code from C++ to Java, helping them understand its functionality by making the code more readable. Two zero-shot prompting frameworks are presented and evaluated for their effectiveness, achieving syntactically correct outputs at rates between 62.2% and 63.9%. Of these outputs, between 39.0% and 42.9% produce functionally preserved translations.
This research focuses on the use of LLMs as a code translation tool for beginner malware researchers. Novices can use such a tool to translate malware source code from C++ to Java, helping them understand its functionality by making the code more readable. Two zero-shot prompting frameworks are presented and evaluated for their effectiveness, achieving syntactically correct outputs at rates between 62.2% and 63.9%. Of these outputs, between 39.0% and 42.9% produce functionally preserved translations.
Dynamic malware analysis produces large amounts of behavioural evidence, which can be difficult to interpret manually and too large to process directly with small Large Language Models (LLMs). This paper evaluates to what extent Qwen3-4B can distinguish between benign and malicious Windows executables using reduced CAPEv2 dynamic-analysis reports. To test this, we built a sandbox pipeline which executes samples in a Windows 10 Pro detonation VM, collects CAPEv2 reports, filters them down to the most relevant dynamic-analysis information, and feeds them to Qwen for classification. The reduced reports retain the process tree, domains and DNS activity, behavioural signatures, and their ATT&CK TTP and Malware Behavior Catalog mappings. The dataset consisted of 1082 malware samples from MalwareBazaar and 762 benign samples collected from PortableApps, PortableApps installers, the Sysinternals Suite, and Benign-NET. Two prompts were tested: one with only benign and malware as possible verdicts, and one which also allowed an inconclusive verdict. The first prompt achieved 73.01% recall and 60.91% precision, showing that Qwen could detect many malware samples but also misclassified many benign samples as malicious. The second prompt did not solve this issue, since more correct classifications became inconclusive than incorrect ones. Overall, Qwen3-4B shows some potential for dynamic malware analysis, but its high false positive rate makes it unsuitable as a standalone classifier without further improvements or fine-tuning.
...
Dynamic malware analysis produces large amounts of behavioural evidence, which can be difficult to interpret manually and too large to process directly with small Large Language Models (LLMs). This paper evaluates to what extent Qwen3-4B can distinguish between benign and malicious Windows executables using reduced CAPEv2 dynamic-analysis reports. To test this, we built a sandbox pipeline which executes samples in a Windows 10 Pro detonation VM, collects CAPEv2 reports, filters them down to the most relevant dynamic-analysis information, and feeds them to Qwen for classification. The reduced reports retain the process tree, domains and DNS activity, behavioural signatures, and their ATT&CK TTP and Malware Behavior Catalog mappings. The dataset consisted of 1082 malware samples from MalwareBazaar and 762 benign samples collected from PortableApps, PortableApps installers, the Sysinternals Suite, and Benign-NET. Two prompts were tested: one with only benign and malware as possible verdicts, and one which also allowed an inconclusive verdict. The first prompt achieved 73.01% recall and 60.91% precision, showing that Qwen could detect many malware samples but also misclassified many benign samples as malicious. The second prompt did not solve this issue, since more correct classifications became inconclusive than incorrect ones. Overall, Qwen3-4B shows some potential for dynamic malware analysis, but its high false positive rate makes it unsuitable as a standalone classifier without further improvements or fine-tuning.
Anytime Program Synthesis for the BEN ARC Solver
Exploring heuristically-driven backtracking for the DA&C paradigm
Program synthesis for the Abstraction and Reasoning Corpus (ARC) remains challenging due to large search spaces, limited computational budgets, and the propagation of early synthesis decisions throughout the solver pipeline. This paper investigates whether internal solver metrics can guide retriggerable transformation search within the BEN Divide, Align, and Conquer (DA&C) architecture.To address this question, we introduce an anytime transformation-refinement mechanism that resumes synthesis from previously explored search frontiers, along with a threshold-based retriggering strategy driven by transformation overfitting and solver-state metrics. The proposed approach was implemented in a Julia-based reimplementation of BEN using the Herb.jl program synthesis framework and evaluated on the ARC benchmark.The experimental results show that retriggerable synthesis substantially changes the distribution of search effort across correspondences, enabling deeper exploration of highly ranked alignments while preserving exploration of lower-ranked alternatives. Compared to the baseline solver, the proposed method explores significantly fewer candidate transformations. It produces a different set of solved tasks, demonstrating that it redirects synthesis toward different regions of the search space. However, these changes do not translate into a large improvement in overall ARC solve rate, with both approaches achieving comparable aggregate performance.These findings suggest that transformation synthesis is not the primary bottleneck in the current BEN architecture. Instead, overall performance appears to be constrained by earlier stages of the pipeline, including segmentation, correspondence generation, and the expressiveness of the transformation language. While retriggerable anytime synthesis does not increase aggregate benchmark runtime performance, it provides a structured mechanism for adaptive search control. It lays the foundation for future work on stateful, dynamically allocated program synthesis.
...
Program synthesis for the Abstraction and Reasoning Corpus (ARC) remains challenging due to large search spaces, limited computational budgets, and the propagation of early synthesis decisions throughout the solver pipeline. This paper investigates whether internal solver metrics can guide retriggerable transformation search within the BEN Divide, Align, and Conquer (DA&C) architecture.To address this question, we introduce an anytime transformation-refinement mechanism that resumes synthesis from previously explored search frontiers, along with a threshold-based retriggering strategy driven by transformation overfitting and solver-state metrics. The proposed approach was implemented in a Julia-based reimplementation of BEN using the Herb.jl program synthesis framework and evaluated on the ARC benchmark.The experimental results show that retriggerable synthesis substantially changes the distribution of search effort across correspondences, enabling deeper exploration of highly ranked alignments while preserving exploration of lower-ranked alternatives. Compared to the baseline solver, the proposed method explores significantly fewer candidate transformations. It produces a different set of solved tasks, demonstrating that it redirects synthesis toward different regions of the search space. However, these changes do not translate into a large improvement in overall ARC solve rate, with both approaches achieving comparable aggregate performance.These findings suggest that transformation synthesis is not the primary bottleneck in the current BEN architecture. Instead, overall performance appears to be constrained by earlier stages of the pipeline, including segmentation, correspondence generation, and the expressiveness of the transformation language. While retriggerable anytime synthesis does not increase aggregate benchmark runtime performance, it provides a structured mechanism for adaptive search control. It lays the foundation for future work on stateful, dynamically allocated program synthesis.
Efficient learning through programmatic representations
Improving transformation search in BEN
The Abstraction and Reasoning Corpus (ARC) challenges systems to recognise patterns from just a few input-output examples, which proves to be a setting where most current approaches struggle. Program synthesis systems like BEN tackle this by decomposing the images into objects rather than pixels and then searching for transformation programs that explain the examples. However, its reliance on exhaustive enumeration makes the transformation search computationally expensive and difficult to scale.
This project aims to improve the transformation search and, in doing so, asks whether correspondence-tailored grammar pruning can reduce the search space and improve the efficiency of BEN’s conquer step. BEN is reimplemented in Julia using the Herb.jl library. On top of that, a pruning strategy that exploits structural similarities between matched input-output object pairs is added. Both the baseline and the improved version are evaluated on the 400-task ARC training set.
The pruning reduces the average number of candidate programs evaluated by 77%, but it has a seemingly slight negative impact on the tasks that get solved. The number of correctly solved tasks decreases from 31 to 30. These results show that simple structural observations about the matched object pairs substantially reduce the search space. More informed search looks like a promising direction for improving program synthesis systems on ARC. ...
This project aims to improve the transformation search and, in doing so, asks whether correspondence-tailored grammar pruning can reduce the search space and improve the efficiency of BEN’s conquer step. BEN is reimplemented in Julia using the Herb.jl library. On top of that, a pruning strategy that exploits structural similarities between matched input-output object pairs is added. Both the baseline and the improved version are evaluated on the 400-task ARC training set.
The pruning reduces the average number of candidate programs evaluated by 77%, but it has a seemingly slight negative impact on the tasks that get solved. The number of correctly solved tasks decreases from 31 to 30. These results show that simple structural observations about the matched object pairs substantially reduce the search space. More informed search looks like a promising direction for improving program synthesis systems on ARC. ...
The Abstraction and Reasoning Corpus (ARC) challenges systems to recognise patterns from just a few input-output examples, which proves to be a setting where most current approaches struggle. Program synthesis systems like BEN tackle this by decomposing the images into objects rather than pixels and then searching for transformation programs that explain the examples. However, its reliance on exhaustive enumeration makes the transformation search computationally expensive and difficult to scale.
This project aims to improve the transformation search and, in doing so, asks whether correspondence-tailored grammar pruning can reduce the search space and improve the efficiency of BEN’s conquer step. BEN is reimplemented in Julia using the Herb.jl library. On top of that, a pruning strategy that exploits structural similarities between matched input-output object pairs is added. Both the baseline and the improved version are evaluated on the 400-task ARC training set.
The pruning reduces the average number of candidate programs evaluated by 77%, but it has a seemingly slight negative impact on the tasks that get solved. The number of correctly solved tasks decreases from 31 to 30. These results show that simple structural observations about the matched object pairs substantially reduce the search space. More informed search looks like a promising direction for improving program synthesis systems on ARC.
This project aims to improve the transformation search and, in doing so, asks whether correspondence-tailored grammar pruning can reduce the search space and improve the efficiency of BEN’s conquer step. BEN is reimplemented in Julia using the Herb.jl library. On top of that, a pruning strategy that exploits structural similarities between matched input-output object pairs is added. Both the baseline and the improved version are evaluated on the 400-task ARC training set.
The pruning reduces the average number of candidate programs evaluated by 77%, but it has a seemingly slight negative impact on the tasks that get solved. The number of correctly solved tasks decreases from 31 to 30. These results show that simple structural observations about the matched object pairs substantially reduce the search space. More informed search looks like a promising direction for improving program synthesis systems on ARC.
Efficient learning through programmatic representations
How can we better find the matches between input and output objects?
The Abstraction and Reasoning Corpus (ARC) serves as a challenging benchmark designed to measure human-like artificial intelligence by only providing a few training examples for each task. Program synthesis offers a new approach to solving this benchmark by generating rule-based transformation programs. A recent approach, BEN, tackles these tasks by making use of program synthesis in a Divide, Align and Conquer strategy. The Align component identifies correspondences between the input and output objects by using a Structure Mapping Engine (SME) rooted in analogical reasoning. Despite its success, each individual part of the algorithm can be further improved, in particular the features of objects used in Align.
We present an enhanced object-matching methodology of the Align component to improve the quality of the correspondences found. First, the BEN algorithm is re-implemented in Julia, making use of an Answer Set Programming (ASP) solver using Clingo and Prolog for the SME to offer better efficiency. Second, we augment the structural representation of the objects by introducing features that capture the spatial relations between them, specifically through forms of ranked coordinates. Furthermore, we refine how multi-coloured objects are propositionally encoded. Finally, we introduce a weighting heuristic for the features: the significance of individual visual attributes is minimized when the input and output grids contain the same number of objects, and simple coordinates are eliminated when the input and output grids have different sizes.
The proposed changes were evaluated by isolating the Align component across subsets of the ARC-AGI-1 benchmark. In a sample of 25 manually selected tasks, the number of perfectly matched tasks improved significantly from 11 to 23. In a randomly selected sample of 25 tasks, the modifications give better or identical matches in 22 tasks, with only 3 showing degradations. When running the entire benchmark with the full BEN algorithm, one extra task is solved and two no longer are. Average times are similar and the number of searched transformations decreases. These findings suggest that including spatial relations and contextual weighting of the features improves the accuracy of finding correct correspondences for the ARC benchmark. ...
We present an enhanced object-matching methodology of the Align component to improve the quality of the correspondences found. First, the BEN algorithm is re-implemented in Julia, making use of an Answer Set Programming (ASP) solver using Clingo and Prolog for the SME to offer better efficiency. Second, we augment the structural representation of the objects by introducing features that capture the spatial relations between them, specifically through forms of ranked coordinates. Furthermore, we refine how multi-coloured objects are propositionally encoded. Finally, we introduce a weighting heuristic for the features: the significance of individual visual attributes is minimized when the input and output grids contain the same number of objects, and simple coordinates are eliminated when the input and output grids have different sizes.
The proposed changes were evaluated by isolating the Align component across subsets of the ARC-AGI-1 benchmark. In a sample of 25 manually selected tasks, the number of perfectly matched tasks improved significantly from 11 to 23. In a randomly selected sample of 25 tasks, the modifications give better or identical matches in 22 tasks, with only 3 showing degradations. When running the entire benchmark with the full BEN algorithm, one extra task is solved and two no longer are. Average times are similar and the number of searched transformations decreases. These findings suggest that including spatial relations and contextual weighting of the features improves the accuracy of finding correct correspondences for the ARC benchmark. ...
The Abstraction and Reasoning Corpus (ARC) serves as a challenging benchmark designed to measure human-like artificial intelligence by only providing a few training examples for each task. Program synthesis offers a new approach to solving this benchmark by generating rule-based transformation programs. A recent approach, BEN, tackles these tasks by making use of program synthesis in a Divide, Align and Conquer strategy. The Align component identifies correspondences between the input and output objects by using a Structure Mapping Engine (SME) rooted in analogical reasoning. Despite its success, each individual part of the algorithm can be further improved, in particular the features of objects used in Align.
We present an enhanced object-matching methodology of the Align component to improve the quality of the correspondences found. First, the BEN algorithm is re-implemented in Julia, making use of an Answer Set Programming (ASP) solver using Clingo and Prolog for the SME to offer better efficiency. Second, we augment the structural representation of the objects by introducing features that capture the spatial relations between them, specifically through forms of ranked coordinates. Furthermore, we refine how multi-coloured objects are propositionally encoded. Finally, we introduce a weighting heuristic for the features: the significance of individual visual attributes is minimized when the input and output grids contain the same number of objects, and simple coordinates are eliminated when the input and output grids have different sizes.
The proposed changes were evaluated by isolating the Align component across subsets of the ARC-AGI-1 benchmark. In a sample of 25 manually selected tasks, the number of perfectly matched tasks improved significantly from 11 to 23. In a randomly selected sample of 25 tasks, the modifications give better or identical matches in 22 tasks, with only 3 showing degradations. When running the entire benchmark with the full BEN algorithm, one extra task is solved and two no longer are. Average times are similar and the number of searched transformations decreases. These findings suggest that including spatial relations and contextual weighting of the features improves the accuracy of finding correct correspondences for the ARC benchmark.
We present an enhanced object-matching methodology of the Align component to improve the quality of the correspondences found. First, the BEN algorithm is re-implemented in Julia, making use of an Answer Set Programming (ASP) solver using Clingo and Prolog for the SME to offer better efficiency. Second, we augment the structural representation of the objects by introducing features that capture the spatial relations between them, specifically through forms of ranked coordinates. Furthermore, we refine how multi-coloured objects are propositionally encoded. Finally, we introduce a weighting heuristic for the features: the significance of individual visual attributes is minimized when the input and output grids contain the same number of objects, and simple coordinates are eliminated when the input and output grids have different sizes.
The proposed changes were evaluated by isolating the Align component across subsets of the ARC-AGI-1 benchmark. In a sample of 25 manually selected tasks, the number of perfectly matched tasks improved significantly from 11 to 23. In a randomly selected sample of 25 tasks, the modifications give better or identical matches in 22 tasks, with only 3 showing degradations. When running the entire benchmark with the full BEN algorithm, one extra task is solved and two no longer are. Average times are similar and the number of searched transformations decreases. These findings suggest that including spatial relations and contextual weighting of the features improves the accuracy of finding correct correspondences for the ARC benchmark.
The introduction of large language models (LLMs) has transformed the way software is written. With the help of LLM powered code generation the productivity of software engineers has increased all over the world. However, these models are also computationally expensive. The ubiquitous use of these models has raised significant sustainability concerns.
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial. ...
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial. ...
The introduction of large language models (LLMs) has transformed the way software is written. With the help of LLM powered code generation the productivity of software engineers has increased all over the world. However, these models are also computationally expensive. The ubiquitous use of these models has raised significant sustainability concerns.
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial.
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial.
Automated program repair (APR) is increasingly critical in modern software development, yet language models (LMs) often struggle to capture repository-specific conventions and constraints. Small language models (SLMs) offer a cost-effective and deployable alternative, but their performance depends heavily on high-quality domain-specific supervision. In this work, we introduce a multi-teacher distillation pipeline that generates multi-turn repair trajectories, including both successful fixes and intermediate failures, to construct rich training datasets for method-level APR. We systematically analyze the impact of dataset size, repair diversity, fine-tuning strategies, hyperparameters, and reasoning supervision, aiming to identify efficient and reliable approaches for adapting SLMs to repository-specific repair tasks.
Our experiments demonstrate that parameter-efficient fine-tuning, particularly LoRA with carefully selected adapter ranks, achieves strong performance across reasoning and non-reasoning regimes while maintaining low computational cost. Explicit reasoning supervision is not required for high repair accuracy, but it significantly reduces reasoning trace lengths and inference costs. Dataset diversity and multi-turn trajectories are key to improving generalization and bridging the gap between reasoning and non-reasoning inference. Finally, this study seeks to provide empirical insights into the practical adaptation of SLMs for repository-specific APR, evaluating how strategic choices in dataset design, lightweight fine-tuning approaches, and reasoning supervision influence performance in real-world contexts. ...
Our experiments demonstrate that parameter-efficient fine-tuning, particularly LoRA with carefully selected adapter ranks, achieves strong performance across reasoning and non-reasoning regimes while maintaining low computational cost. Explicit reasoning supervision is not required for high repair accuracy, but it significantly reduces reasoning trace lengths and inference costs. Dataset diversity and multi-turn trajectories are key to improving generalization and bridging the gap between reasoning and non-reasoning inference. Finally, this study seeks to provide empirical insights into the practical adaptation of SLMs for repository-specific APR, evaluating how strategic choices in dataset design, lightweight fine-tuning approaches, and reasoning supervision influence performance in real-world contexts. ...
Automated program repair (APR) is increasingly critical in modern software development, yet language models (LMs) often struggle to capture repository-specific conventions and constraints. Small language models (SLMs) offer a cost-effective and deployable alternative, but their performance depends heavily on high-quality domain-specific supervision. In this work, we introduce a multi-teacher distillation pipeline that generates multi-turn repair trajectories, including both successful fixes and intermediate failures, to construct rich training datasets for method-level APR. We systematically analyze the impact of dataset size, repair diversity, fine-tuning strategies, hyperparameters, and reasoning supervision, aiming to identify efficient and reliable approaches for adapting SLMs to repository-specific repair tasks.
Our experiments demonstrate that parameter-efficient fine-tuning, particularly LoRA with carefully selected adapter ranks, achieves strong performance across reasoning and non-reasoning regimes while maintaining low computational cost. Explicit reasoning supervision is not required for high repair accuracy, but it significantly reduces reasoning trace lengths and inference costs. Dataset diversity and multi-turn trajectories are key to improving generalization and bridging the gap between reasoning and non-reasoning inference. Finally, this study seeks to provide empirical insights into the practical adaptation of SLMs for repository-specific APR, evaluating how strategic choices in dataset design, lightweight fine-tuning approaches, and reasoning supervision influence performance in real-world contexts.
Our experiments demonstrate that parameter-efficient fine-tuning, particularly LoRA with carefully selected adapter ranks, achieves strong performance across reasoning and non-reasoning regimes while maintaining low computational cost. Explicit reasoning supervision is not required for high repair accuracy, but it significantly reduces reasoning trace lengths and inference costs. Dataset diversity and multi-turn trajectories are key to improving generalization and bridging the gap between reasoning and non-reasoning inference. Finally, this study seeks to provide empirical insights into the practical adaptation of SLMs for repository-specific APR, evaluating how strategic choices in dataset design, lightweight fine-tuning approaches, and reasoning supervision influence performance in real-world contexts.
Search-Based Software Testing (SBST) tools can automatically generate tests to achieve high code coverage; however, a systematic understanding of why they fail in specific situations is necessary. This thesis addresses this gap by developing a comprehensive taxonomy of coverage failures through an empirical analysis of the three most prominent SBST tools: Pynguin (Python), SynTest (JavaScript), and EvoSuite (Java). By classifying and analysing failure patterns across these tools and language paradigms, this research provides a foundational framework to diagnose shortcomings, prioritise future development, and enhance the practical effectiveness of automated test generation.
...
Search-Based Software Testing (SBST) tools can automatically generate tests to achieve high code coverage; however, a systematic understanding of why they fail in specific situations is necessary. This thesis addresses this gap by developing a comprehensive taxonomy of coverage failures through an empirical analysis of the three most prominent SBST tools: Pynguin (Python), SynTest (JavaScript), and EvoSuite (Java). By classifying and analysing failure patterns across these tools and language paradigms, this research provides a foundational framework to diagnose shortcomings, prioritise future development, and enhance the practical effectiveness of automated test generation.
Most classical Byzantine agreement protocols between n nodes offer a fault tolerance t of up to t < n/3. Quantum fault-tolerant consensus protocols have been proposed that achieve a tolerance of t < n/2 and are therefore worth studying. In this paper, we assess the failure probability of a previously proposed quantum-aided weak broad- cast protocol and how it is affected by a physical error source. Specifically, we study the effect of measurement error noise. We simulate the protocol as-is on a four-node quantum network composed of NV-center devices, both with zero and with one faulty node. We then apply a measurement error noise model and compare the failure prob- ability with the noiseless version. The noise "strength" is also varied in order to assess whether the failure probability can be reduced using improved hardware. We show that measurement errors have a significant negative effect on the failure probability of the protocol. In fact, even with 10x improved hardware parameters, the protocol does not achieve an acceptable failure probability. ...
Most classical Byzantine agreement protocols between n nodes offer a fault tolerance t of up to t < n/3. Quantum fault-tolerant consensus protocols have been proposed that achieve a tolerance of t < n/2 and are therefore worth studying. In this paper, we assess the failure probability of a previously proposed quantum-aided weak broad- cast protocol and how it is affected by a physical error source. Specifically, we study the effect of measurement error noise. We simulate the protocol as-is on a four-node quantum network composed of NV-center devices, both with zero and with one faulty node. We then apply a measurement error noise model and compare the failure prob- ability with the noiseless version. The noise "strength" is also varied in order to assess whether the failure probability can be reduced using improved hardware. We show that measurement errors have a significant negative effect on the failure probability of the protocol. In fact, even with 10x improved hardware parameters, the protocol does not achieve an acceptable failure probability.
Noisy Byzantine Agreement Protocol in a Small Quantum Network
The Failure Probability of the Protocol Under Leakage Errors
The Byzantine agreement problem is a challenge in distributed computing that explores how reliable parties can reach consensus even though there are unreliable or malicious participants. Quantum solutions propose better fault tolerance than the classical solutions, tolerating up to t < n/2 number of faulty or malicious parties, compared to the classical solutions with a limit t < n/3 where n is the number of participating parties. The Weak Broadcast Protocol we study proposes a quantum-aided solution using entangled four-qubit states. However, the hardware imperfections of quantum computers affect the reliability of the protocol. This paper investigates the impact of leakage errors, defined as unintended measurement outcomes that fall outside the set of expected basis states, on the failure probability of the protocol. We simulate the protocol in a noise-free setting using SquidASM to validate our setup and failure probabilities against the results reported in prior work. We then introduce a custom bit-flip leakage noise model, supported by an analytical formulation, and compare it with the pessimistic assumptions on the effect of the leakage errors by the prior work. Our results show that while the protocol’s failure probability increases significantly under leakage noise, it is not as pessimistic as assumed before. The observed behavior differs for different fault configurations and shows that moderate noise may significantly affect the failure probability of the protocol. These findings suggest that the protocol is vulnerable to realistic noise.
...
The Byzantine agreement problem is a challenge in distributed computing that explores how reliable parties can reach consensus even though there are unreliable or malicious participants. Quantum solutions propose better fault tolerance than the classical solutions, tolerating up to t < n/2 number of faulty or malicious parties, compared to the classical solutions with a limit t < n/3 where n is the number of participating parties. The Weak Broadcast Protocol we study proposes a quantum-aided solution using entangled four-qubit states. However, the hardware imperfections of quantum computers affect the reliability of the protocol. This paper investigates the impact of leakage errors, defined as unintended measurement outcomes that fall outside the set of expected basis states, on the failure probability of the protocol. We simulate the protocol in a noise-free setting using SquidASM to validate our setup and failure probabilities against the results reported in prior work. We then introduce a custom bit-flip leakage noise model, supported by an analytical formulation, and compare it with the pessimistic assumptions on the effect of the leakage errors by the prior work. Our results show that while the protocol’s failure probability increases significantly under leakage noise, it is not as pessimistic as assumed before. The observed behavior differs for different fault configurations and shows that moderate noise may significantly affect the failure probability of the protocol. These findings suggest that the protocol is vulnerable to realistic noise.
Noisy Byzantine Agreement in Quantum Networks
Impact of Gate Errors on a Weak Broadcast Protocol
The Weak Broadcast protocol is a quantum solution to the Byzantine agreement problem. However, its practical applicability remains uncertain due to the impact of noise in quantum systems. Building on prior work by Guba et al., which presented a quantum Byzantine agreement protocol using a four-qubit singlet state, this research investigates how gate-level noise affects the success probability of the protocol, focusing on a Linear Circuit implementation. Gate noise is a critical challenge in quantum hardware, as quantum gates are a primary source of errors in current devices. Using the NetSquid and SquidASM frameworks, the protocol was reproduced and extended with a realistic depolarizing noise model to simulate noisy conditions across different scenarios. Results show that even modest noise levels (0.001% - 0.01%) lead to a sharp rise in failure probability, and at 1% noise, the protocol fails often. These findings highlight the protocol's vulnerability to gate noise and suggest that its practical deployment would require significant error mitigation and fault-tolerant architectures, or a design adapted to better tolerate gate-level noise. This work offers a reproducible simulation framework and provides insights into the protocol's robustness under noisy conditions, addressing a gap in current literature.
...
The Weak Broadcast protocol is a quantum solution to the Byzantine agreement problem. However, its practical applicability remains uncertain due to the impact of noise in quantum systems. Building on prior work by Guba et al., which presented a quantum Byzantine agreement protocol using a four-qubit singlet state, this research investigates how gate-level noise affects the success probability of the protocol, focusing on a Linear Circuit implementation. Gate noise is a critical challenge in quantum hardware, as quantum gates are a primary source of errors in current devices. Using the NetSquid and SquidASM frameworks, the protocol was reproduced and extended with a realistic depolarizing noise model to simulate noisy conditions across different scenarios. Results show that even modest noise levels (0.001% - 0.01%) lead to a sharp rise in failure probability, and at 1% noise, the protocol fails often. These findings highlight the protocol's vulnerability to gate noise and suggest that its practical deployment would require significant error mitigation and fault-tolerant architectures, or a design adapted to better tolerate gate-level noise. This work offers a reproducible simulation framework and provides insights into the protocol's robustness under noisy conditions, addressing a gap in current literature.
Quantum Byzantine Agreement Protocol Under Noisy Conditions
Evaluating the Impact of Qubit Decoherence on the Protocol’s Success Rate
Byzantine agreement protocols allow distributed systems to achieve consensus despite faulty nodes. The classical solution to the Byzantine agreement can only tolerate less than 1/3 of the total nodes being faulty, while a quantum-aided protocol has the potential to tolerate up to 1/2 of the nodes being faulty. However, the quantum states used in the protocol are vulnerable to decoherence, a process that results in the degradation of a quantum state. This research investigates how memory decoherence affects the failure rate of a three-party quantum Byzantine agreement protocol. The primary contribution is demonstrating that the protocol is resilient to carbon T2 decoherence, as its Z-basis measurement scheme is insensitive to phase errors. Conversely, the failure probability increases with decreasing carbon T1 decoherence values. Extrapolating from the observed trend of decreasing failure with longer T1 coherence times, the protocol’s performance on nitrogen vacancy center hardware is expected to approach the ideal, noiseless limit due to their experimentally established long T1 times.
...
Byzantine agreement protocols allow distributed systems to achieve consensus despite faulty nodes. The classical solution to the Byzantine agreement can only tolerate less than 1/3 of the total nodes being faulty, while a quantum-aided protocol has the potential to tolerate up to 1/2 of the nodes being faulty. However, the quantum states used in the protocol are vulnerable to decoherence, a process that results in the degradation of a quantum state. This research investigates how memory decoherence affects the failure rate of a three-party quantum Byzantine agreement protocol. The primary contribution is demonstrating that the protocol is resilient to carbon T2 decoherence, as its Z-basis measurement scheme is insensitive to phase errors. Conversely, the failure probability increases with decreasing carbon T1 decoherence values. Extrapolating from the observed trend of decreasing failure with longer T1 coherence times, the protocol’s performance on nitrogen vacancy center hardware is expected to approach the ideal, noiseless limit due to their experimentally established long T1 times.
Large Language Models (LLMs) are increasingly used for code-centric tasks. However, their training data often exhibits data smells that may hinder downstream quality. This research focuses on the “Uneven Natural Languages” smell and the presence of non-English text in source code and investigates its effect on LLM-based code generation and summarisation. We construct a three-stage (Detection, Generation, Evaluation) pipeline that annotates every character in a file with its predicted language using Tree-sitter, FastText, and pycld2; masks target spans via causal masking and Fill-in-the-Middle (FIM) and prompts using three chosen models (SmolLM2, StarCoder 2, and Mellum-4B). The Heap dataset is used for the pipeline; however, this research only focuses on the Java subset of the Heap.
In 3.35 million Java files, we find that English tokens account for more than 90\% of comments, strings, and identifiers, while Chinese, Spanish, Portuguese, and French form a long-tailed minority. Despite this skew, LLMs achieve marginally higher BLEU, METEOR, ROUGE, and Exact Match scores when non-English elements are present or masked. Mellum consistently yields the most fluent continuations; StarCoder 2 retains broader token recall; SmolLM2 lags on both axes, reflecting its smaller capacity.
Our publicly available code enables reproducible assessment of multilingual data smells and lays the groundwork for cleaner, language-aware pre-training corpora and more robust multilingual code assistants. ...
In 3.35 million Java files, we find that English tokens account for more than 90\% of comments, strings, and identifiers, while Chinese, Spanish, Portuguese, and French form a long-tailed minority. Despite this skew, LLMs achieve marginally higher BLEU, METEOR, ROUGE, and Exact Match scores when non-English elements are present or masked. Mellum consistently yields the most fluent continuations; StarCoder 2 retains broader token recall; SmolLM2 lags on both axes, reflecting its smaller capacity.
Our publicly available code enables reproducible assessment of multilingual data smells and lays the groundwork for cleaner, language-aware pre-training corpora and more robust multilingual code assistants. ...
Large Language Models (LLMs) are increasingly used for code-centric tasks. However, their training data often exhibits data smells that may hinder downstream quality. This research focuses on the “Uneven Natural Languages” smell and the presence of non-English text in source code and investigates its effect on LLM-based code generation and summarisation. We construct a three-stage (Detection, Generation, Evaluation) pipeline that annotates every character in a file with its predicted language using Tree-sitter, FastText, and pycld2; masks target spans via causal masking and Fill-in-the-Middle (FIM) and prompts using three chosen models (SmolLM2, StarCoder 2, and Mellum-4B). The Heap dataset is used for the pipeline; however, this research only focuses on the Java subset of the Heap.
In 3.35 million Java files, we find that English tokens account for more than 90\% of comments, strings, and identifiers, while Chinese, Spanish, Portuguese, and French form a long-tailed minority. Despite this skew, LLMs achieve marginally higher BLEU, METEOR, ROUGE, and Exact Match scores when non-English elements are present or masked. Mellum consistently yields the most fluent continuations; StarCoder 2 retains broader token recall; SmolLM2 lags on both axes, reflecting its smaller capacity.
Our publicly available code enables reproducible assessment of multilingual data smells and lays the groundwork for cleaner, language-aware pre-training corpora and more robust multilingual code assistants.
In 3.35 million Java files, we find that English tokens account for more than 90\% of comments, strings, and identifiers, while Chinese, Spanish, Portuguese, and French form a long-tailed minority. Despite this skew, LLMs achieve marginally higher BLEU, METEOR, ROUGE, and Exact Match scores when non-English elements are present or masked. Mellum consistently yields the most fluent continuations; StarCoder 2 retains broader token recall; SmolLM2 lags on both axes, reflecting its smaller capacity.
Our publicly available code enables reproducible assessment of multilingual data smells and lays the groundwork for cleaner, language-aware pre-training corpora and more robust multilingual code assistants.