AS
A. Sandu
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
LLM-guided Concurrency Testing
Can AI coding agents enhance sampling-based CCT schedulers?
Controlled Concurrency Testing (CCT) schedulers explore a program’s
thread interleavings to expose concurrency bugs, but operate
without upfront knowledge of its concurrency design. Samplingbased
scheduling algorithms such as Selective Uniform Random
Walk (SURW) use a set of interesting events to focus their exploration,
but identifying these events currently requires manual expert
analysis or random sampling. Towards that end, we aim to
investigate the applicability of LLMs in analysis for CCT. We first
survey the literature on LLMs for concurrency analysis, finding
that current models reliably identify concurrency primitives but
struggle with complex interleavings, motivating the use of an agent
for the coarser task of identifying relevant resources rather than
detecting bugs directly. We then propose DeltaScout, a pipeline in
which an AI coding agent autonomously explores a Java project and
generates this set of interesting events before testing begins. The
agent reads source files, optionally queries a statically constructed
call graph and reasons about shared resources to produce a heuristic
for interesting events used by SURW (SURW+Agent), alongside a
hybrid variant that combines LLM-selected and randomly sampled
events (SURW-Hybrid+Agent).
We integrate DeltaScout within a state-of-the-art JVM CCT
framework, and evaluate them against SURW (random Δ) and a
naive random scheduler baseline on SCTBench, JaConTeBe and
Apache Kafka Streams. On SCTBench, our approach finds all 28
bugs, including one timing-sensitive atomicity violation that SURW
(random Δ) misses. On JaConTeBe, SURW (random Δ) finds 15/25
and SURW+Agent finds 14/25; the hybrid variant matches the baseline
at 15/25. On Kafka, no LLM configuration finds more confirmed
bugs than the SURW baseline (8 of 9), but the best configurations tie
it on a complementary set: only the baseline triggers one state race,
while only our approach triggers a thread leak no baseline scheduler
reaches. Through an ablation study, we find that a call-graph
navigation tool and a semantic context document are synergistic,
as neither alone improves bug detection. We also evaluate Claude
Sonnet 4.6, Opus 4.8 and Haiku 4.5 as the underlying model, finding
that Sonnet performs best.We find that LLM-based agents are capable
of contributing to the field of CCT, but that raw bug detection
counts do not improve substantially. Rather, the time to find some
more "difficult" bugs is much shorter and a bug is found which is
unable to be triggered by the baseline. ...
thread interleavings to expose concurrency bugs, but operate
without upfront knowledge of its concurrency design. Samplingbased
scheduling algorithms such as Selective Uniform Random
Walk (SURW) use a set of interesting events to focus their exploration,
but identifying these events currently requires manual expert
analysis or random sampling. Towards that end, we aim to
investigate the applicability of LLMs in analysis for CCT. We first
survey the literature on LLMs for concurrency analysis, finding
that current models reliably identify concurrency primitives but
struggle with complex interleavings, motivating the use of an agent
for the coarser task of identifying relevant resources rather than
detecting bugs directly. We then propose DeltaScout, a pipeline in
which an AI coding agent autonomously explores a Java project and
generates this set of interesting events before testing begins. The
agent reads source files, optionally queries a statically constructed
call graph and reasons about shared resources to produce a heuristic
for interesting events used by SURW (SURW+Agent), alongside a
hybrid variant that combines LLM-selected and randomly sampled
events (SURW-Hybrid+Agent).
We integrate DeltaScout within a state-of-the-art JVM CCT
framework, and evaluate them against SURW (random Δ) and a
naive random scheduler baseline on SCTBench, JaConTeBe and
Apache Kafka Streams. On SCTBench, our approach finds all 28
bugs, including one timing-sensitive atomicity violation that SURW
(random Δ) misses. On JaConTeBe, SURW (random Δ) finds 15/25
and SURW+Agent finds 14/25; the hybrid variant matches the baseline
at 15/25. On Kafka, no LLM configuration finds more confirmed
bugs than the SURW baseline (8 of 9), but the best configurations tie
it on a complementary set: only the baseline triggers one state race,
while only our approach triggers a thread leak no baseline scheduler
reaches. Through an ablation study, we find that a call-graph
navigation tool and a semantic context document are synergistic,
as neither alone improves bug detection. We also evaluate Claude
Sonnet 4.6, Opus 4.8 and Haiku 4.5 as the underlying model, finding
that Sonnet performs best.We find that LLM-based agents are capable
of contributing to the field of CCT, but that raw bug detection
counts do not improve substantially. Rather, the time to find some
more "difficult" bugs is much shorter and a bug is found which is
unable to be triggered by the baseline. ...
Controlled Concurrency Testing (CCT) schedulers explore a program’s
thread interleavings to expose concurrency bugs, but operate
without upfront knowledge of its concurrency design. Samplingbased
scheduling algorithms such as Selective Uniform Random
Walk (SURW) use a set of interesting events to focus their exploration,
but identifying these events currently requires manual expert
analysis or random sampling. Towards that end, we aim to
investigate the applicability of LLMs in analysis for CCT. We first
survey the literature on LLMs for concurrency analysis, finding
that current models reliably identify concurrency primitives but
struggle with complex interleavings, motivating the use of an agent
for the coarser task of identifying relevant resources rather than
detecting bugs directly. We then propose DeltaScout, a pipeline in
which an AI coding agent autonomously explores a Java project and
generates this set of interesting events before testing begins. The
agent reads source files, optionally queries a statically constructed
call graph and reasons about shared resources to produce a heuristic
for interesting events used by SURW (SURW+Agent), alongside a
hybrid variant that combines LLM-selected and randomly sampled
events (SURW-Hybrid+Agent).
We integrate DeltaScout within a state-of-the-art JVM CCT
framework, and evaluate them against SURW (random Δ) and a
naive random scheduler baseline on SCTBench, JaConTeBe and
Apache Kafka Streams. On SCTBench, our approach finds all 28
bugs, including one timing-sensitive atomicity violation that SURW
(random Δ) misses. On JaConTeBe, SURW (random Δ) finds 15/25
and SURW+Agent finds 14/25; the hybrid variant matches the baseline
at 15/25. On Kafka, no LLM configuration finds more confirmed
bugs than the SURW baseline (8 of 9), but the best configurations tie
it on a complementary set: only the baseline triggers one state race,
while only our approach triggers a thread leak no baseline scheduler
reaches. Through an ablation study, we find that a call-graph
navigation tool and a semantic context document are synergistic,
as neither alone improves bug detection. We also evaluate Claude
Sonnet 4.6, Opus 4.8 and Haiku 4.5 as the underlying model, finding
that Sonnet performs best.We find that LLM-based agents are capable
of contributing to the field of CCT, but that raw bug detection
counts do not improve substantially. Rather, the time to find some
more "difficult" bugs is much shorter and a bug is found which is
unable to be triggered by the baseline.
thread interleavings to expose concurrency bugs, but operate
without upfront knowledge of its concurrency design. Samplingbased
scheduling algorithms such as Selective Uniform Random
Walk (SURW) use a set of interesting events to focus their exploration,
but identifying these events currently requires manual expert
analysis or random sampling. Towards that end, we aim to
investigate the applicability of LLMs in analysis for CCT. We first
survey the literature on LLMs for concurrency analysis, finding
that current models reliably identify concurrency primitives but
struggle with complex interleavings, motivating the use of an agent
for the coarser task of identifying relevant resources rather than
detecting bugs directly. We then propose DeltaScout, a pipeline in
which an AI coding agent autonomously explores a Java project and
generates this set of interesting events before testing begins. The
agent reads source files, optionally queries a statically constructed
call graph and reasons about shared resources to produce a heuristic
for interesting events used by SURW (SURW+Agent), alongside a
hybrid variant that combines LLM-selected and randomly sampled
events (SURW-Hybrid+Agent).
We integrate DeltaScout within a state-of-the-art JVM CCT
framework, and evaluate them against SURW (random Δ) and a
naive random scheduler baseline on SCTBench, JaConTeBe and
Apache Kafka Streams. On SCTBench, our approach finds all 28
bugs, including one timing-sensitive atomicity violation that SURW
(random Δ) misses. On JaConTeBe, SURW (random Δ) finds 15/25
and SURW+Agent finds 14/25; the hybrid variant matches the baseline
at 15/25. On Kafka, no LLM configuration finds more confirmed
bugs than the SURW baseline (8 of 9), but the best configurations tie
it on a complementary set: only the baseline triggers one state race,
while only our approach triggers a thread leak no baseline scheduler
reaches. Through an ablation study, we find that a call-graph
navigation tool and a semantic context document are synergistic,
as neither alone improves bug detection. We also evaluate Claude
Sonnet 4.6, Opus 4.8 and Haiku 4.5 as the underlying model, finding
that Sonnet performs best.We find that LLM-based agents are capable
of contributing to the field of CCT, but that raw bug detection
counts do not improve substantially. Rather, the time to find some
more "difficult" bugs is much shorter and a bug is found which is
unable to be triggered by the baseline.
P-STreeD
A Multithreaded Approach for DP Optimal Decision Trees
Decision trees are valued for their ability to logically and transparently classify data. While heuristic methods to compute such trees are efficient, they often compromise on accuracy, prompting interest in Optimal Decision Trees (ODTs), which have the best misclassification score for a given tree size limit and training dataset. The dynamic programming (DP) approach for ODTs has shown improvements over alternatives such as mixed-integer or constraint programming. That being said, it requires further improvements to handle exponential runtime scaling with the depth of the tree and the number of features of the dataset. Leveraging modern hardware, such as multiple CPU cores, offers a promising solution to improve efficiency. This paper proposes a multithreading method for DP ODTs which we apply to STreeD specifically. We introduce a shared memory model and determine which components of the original program can be made local to the threads. We investigate whether it is more efficient to start multithreading at the root of the search tree than near its leaf nodes. We find the former to be superior, resulting in faster runtimes and less shared resource access. Empirical evaluations against the state of the art demonstrate better runtimes, particularly beneficial for large datasets. Finally, thread scaling analyses reveal substantial speed-ups, exceeding 2.5 times with four threads, highlighting our approach's effectiveness for computationally-intensive tasks.
...
Decision trees are valued for their ability to logically and transparently classify data. While heuristic methods to compute such trees are efficient, they often compromise on accuracy, prompting interest in Optimal Decision Trees (ODTs), which have the best misclassification score for a given tree size limit and training dataset. The dynamic programming (DP) approach for ODTs has shown improvements over alternatives such as mixed-integer or constraint programming. That being said, it requires further improvements to handle exponential runtime scaling with the depth of the tree and the number of features of the dataset. Leveraging modern hardware, such as multiple CPU cores, offers a promising solution to improve efficiency. This paper proposes a multithreading method for DP ODTs which we apply to STreeD specifically. We introduce a shared memory model and determine which components of the original program can be made local to the threads. We investigate whether it is more efficient to start multithreading at the root of the search tree than near its leaf nodes. We find the former to be superior, resulting in faster runtimes and less shared resource access. Empirical evaluations against the state of the art demonstrate better runtimes, particularly beneficial for large datasets. Finally, thread scaling analyses reveal substantial speed-ups, exceeding 2.5 times with four threads, highlighting our approach's effectiveness for computationally-intensive tasks.