Circular Image

M.J.G. Olsthoorn

info

Please Note

21 records found

Assessing the Data Efficiency of Masked Autoencoders in Resource-Constrained Environments

Visual foundation models based on Vision Transformers often depend on large datasets and substantial computational resources, limiting their accessibility for resource-constrained research settings. This paper investigates the data efficiency of Masked Autoencoders (MAE) by studying how pre-training dataset size and mask ratio affect downstream representation quality. An MAE model is pre-trained on nested subsets of the same dataset ranging from 1k to 100k images, using different mask ratios, and then evaluated on a different downstream task dataset. The results show that MAE learns transferable representations even from small unlabeled datasets, with downstream accuracy increasing steadily as more pre-training data is used. The experiments also show that the optimal masking difficulty depends on the data regime: lower masking improves validation accuracy for the smallest subsets, while the original 75% MAE mask ratio becomes stronger as the dataset size increases. These findings suggest that mask ratio should not be treated as a fixed default in MAE training. Instead, reducing the mask ratio can improve data efficiency when pre-training data is limited, while higher masking remains effective when more visual variation is available. ...

Data-Efficiency of Self-Supervised Learning with DINO Multi-Crop

Self-supervised learning (SSL) lets computer vision models learn from unlabelled image datasets. Most DINO benchmarks pretrain on ImageNet — a million-image dataset that takes days of multi-GPU training per run, out of reach for the rapid iteration cycles smaller research groups rely on. This leaves practitioners with smaller datasets unsure whether DINO is worth running, or which of its design choices still hold at this scale.

We pretrain a small Vision Transformer (ViT-Tiny/8) using DINO on Tiny-ImageNet subsets from 1K to 100K images at 64x64 resolution, evaluated on downstream classification tasks. Downstream accuracy grows steadily with pretraining-set size and approaches the accuracy of a fully supervised baseline at the largest scale.

Our main contribution is a multi-crop ablation across data scale, training duration, and downstream task category. We find that multi-crop's benefit at sub-ImageNet scale is delayed rather than absent, and that the optimal multi-crop count depends on the downstream task category — no single setting wins across all tasks.

These findings show that the canonical DINO recipe does not transfer cleanly to sub-ImageNet scale. We recommend choosing the multi-crop count based on training budget and downstream task type, rather than copying the ImageNet default. ...

A small-compute characterization with a ViT-Tiny on Tiny-ImageNet subsets

Modern computer vision often reuses a single model, trained once on many images, as a start- ing point for new tasks. Because labels are ex- pensive, a common way to train such a model is self-supervised learning (SSL), which learns from unlabeled images. SSL normally uses millions of images, and it is unclear how well it works when far fewer are available. We study one SSL method, Barlow Twins, in that case. We pre-train a small vision transformer (5.4M parameters) on parts of Tiny-ImageNet, from 1k to 100k unlabeled images, and train every run for the same 1000 epochs, so the only thing that changes is the amount of data. We then freeze each model and measure how well its features transfer to the 19 VTAB-1k tasks. Pre- training helps at every dataset size: the VTAB-1k average rises from 33.7% with 1k images to 39.2% with 100k, well above a 24.4% untrained baseline. But this average hides large differences between tasks: accuracy on natural-image tasks keeps rising with data, while accuracy on more specialized and structured tasks (medical, satellite, and geometric images) changes little. On the smallest dataset, training too long even lowers accuracy. And as the dataset grows, the checkpoint that scores best on the pre-training data moves further from the one that transfers best. At this small scale, then, the amount of data is not the only thing that matters: the kind of downstream task and the checkpoint we keep matter just as much. ...

Data-Efficiency of Self-Supervised Learning with Momentum Contrast

Self-supervised contrastive learning is a popular way to pre-train vision foundation models. So far, it has mostly been studied with large pre-training datasets, and it is most accessible to organizations with massive computational resources. In this work we evaluate the data-efficiency of one such method, Momentum Contrast (MoCo), and investigate how to make it work better when less data is available. We pre-train a Vision Transformer with MoCo on subsets of Tiny-ImageNet ranging from 1,000 to 100,000 images, and evaluate the learned representations on a diverse set of downstream tasks using linear probing. We investigate how the training parameters of MoCo should be chosen for a given amount of data, how the downstream accuracy scales with the amount of pre-training data, and how this scaling differs across types of downstream tasks. We find that the best parameters depend on the amount of data: the optimal number of negatives used for the contrastive objective grows with the size of the dataset, while the momentum coefficient has no single best value. We also find that pre-training is beneficial even with very little data, the downstream accuracy grows approximately log-linearly with the size of the pre-training set, and the data-efficiency growth rate is larger for tasks that are similar to the pre-training data. ...

Deploying the First CoAP-Specific Amplification Honeypot

Bachelor thesis (2026) - A. Kalpakchiev, H.J. Griffioen, M.J.G. Olsthoorn
DDoS amplification attacks exploit connectionless protocols to overwhelm victims
with traffic far exceeding the attacker's own bandwidth. The Constrained Application
Protocol (CoAP), designed for IoT devices, has emerged as an amplification vector
with over 1.7 million exposed reflectors and an amplification factor of approximately
34x, yet no prior work has studied CoAP amplification through honeypot-based
observation. We hypothesize that CoAP's extreme reflector instability - 80\% of
IP addresses change within two weeks - imposes a higher sophistication floor on
adversaries compared to previously studied protocols. To evaluate this, we deploy
the first CoAP-specific amplification honeypot with eight protocol configurations
testing blockwise transfer at four block sizes and rate limiting at three thresholds
across 18 IP addresses for 23 days. The scanning traffic reveals behaviors
undocumented in prior honeypot research: multi-phase CBOR fingerprinting that
distinguishes genuine Eclipse Californium devices from other implementations,
sustained reliability validation over repeated scanning cycles, and efficient list
management where probing ceases after device cataloguing. Notably, our honeypot was
likely filtered out after failing this fingerprinting test - a finding that
contrasts sharply with prior work on other protocols, where attackers were largely
indifferent to honeypot responses. No amplification attacks were observed, preventing
empirical evaluation of the tested defenses. However, our
findings provide the first evidence that CoAP's structural properties demand higher
adversary sophistication than previously studied amplification protocols, and suggest
that threat models for CoAP must account for fingerprinting-capable actors. ...

Observing scan and test, but not execute, on a public Memcached honeypot

Bachelor thesis (2026) - M. Hájek, Harm Griffioen, Mitchell Olsthoorn
The 2018 Memcached amplification attacks set the public DDoSrecord, after which UDP wasdisabled by default and port 11211 widely firewalled. Yet tens of thousands of misconfigured servers remain reachable, leaving open the question of whether, and how, attackers still exploit this residual surface. We address it with a custom, safety-bounded Memcached honeypot deployed on 16 public IPv4 addresses for 22 days. Over the window it recorded 4,261 packets from 465 sources. Traffic is dominated by stats reconnaissance bearing recognisable tooling fingerprints, and 23 sources swept the full address block: the scan and test phases of the established workflow are clearly present. The execute phase, however, never materialised: no source populated the cache, empirical amplification capped at ≈13×, and the only abuse-related traffic was neutralised. The honeypot egressed just 277 KiB in total. Memcached today behaves as a catalogued but dormant amplifier: continuously enumerated and re-measured, but rarely weaponised. ...

A Honeypot-Based Study of Adversary Tactics, Techniques, and Procedures

DNS amplification is one of the most common forms of distributed denial-of-service (DDoS) attack, yet most large-scale measurements of how attackers abuse DNS in practice date back about a decade. This paper revisits the problem by deploying a DNS amplification honeypot on a university research cluster and analysing the traffic it received over an eleven-day period in 2026. The honeypot behaved as an open resolver, forwarded queries to a real upstream resolver so that responses looked genuine, logged every packet, and ratelimited sources to avoid being weaponised. We collected 3,513 packets from 502 unique source addresses and used the logs to study the temporal pattern of the traffic, the geography and networks of the sources, the domains and query types abused, and the behavioural profiles of the sources. We found that the great majority of traffic was internetwide scanning rather than attacks: only five sources crossed a request-count attack threshold, and only one of these showed behaviour consistent with a genuine amplification attack. ANY queries were still present (9.3% of packets) but were no longer dominant, and we observed almost no DNSKEY or NSEC3 queries, which suggests the shift anticipated after RFC 8482 has not clearly occurred in our data. Our most notable finding is a coordinated group of sources that repeatedly issued a single high-amplification query (google.com TXT, amplification factor 28) without ever crossing the attack threshold, showing that a purely count-based classifier misses the most amplification-relevant behaviour. We discuss why the number of observed attacks was lower than expected and release our code and anonymised dataset to support reproducibility. ...

A honeypot-based observation of real-world DRDoS attacks

Bachelor thesis (2026) - D.J. Hill, H.J. Griffioen, M.J.G. Olsthoorn
Distributed Reflective Denial of Service (DRDoS) attacks leveraging the Network Time Protocol (NTP) continue to threaten network infrastructure. Previous research has been done on this topic, but needs to be reevaluated often due to the high-paced changes in cybersecurity. To capture modern adversarial tactics, this study deployed an experimental network framework consisting of 16 emulated, rate-limited NTP honeypots to log real-world attacker telemetry. Logged data was analysed based on location and target, the sequence of requests, and frequency of attack. Ultimately, this work contributes a safe, open-source honeypot framework, establishes a modern empirical baseline of the NTP DRDoS threat landscape, and underscores the critical need for renewed longitudinal research into evolving adversarial tactics. ...

Optimizing I-JEPA for Data Efficiency

Self-supervised learning eliminates the need for image labels to learn meaningful visual representations, but it does not remove the need for large pretraining datasets. This work studies how Image-based Joint-Embedding Predictive Architecture (I-JEPA) behaves when pretraining data is deliberately limited. We train I-JEPA on stratified Tiny ImageNet subsets and evaluate the frozen representations with CIFAR-10 linear probing. The results show a steep improvement from the smallest subsets to the medium-data regime, followed by a plateau around the largest subsets under the standard final-checkpoint protocol. We also test two architectural modifications motivated by I-JEPA's design: reducing predictor capacity, to test whether an over-expressive predictor absorbs the pretext task instead of forcing useful encoder features, and adding shared photometric augmentation, to test whether extra input variation helps in low-data training. The shallow predictor improves transfer at 32k and 64k images but is neutral or harmful at the smallest and largest splits. The augmentation decreased downstream accuracy at 16k and was neutral at 32k. Additional controls---predictor depth sweeps, fixed-update budgets, and intermediate checkpoint analysis---suggest that the largest-split plateau is partly a training-dynamics issue rather than a pure data-efficiency ceiling. A cross-method comparison with Barlow Twins, MoCo, DINO, and MAE under the shared protocol contextualizes I-JEPA's data efficiency among SSL alternatives. ...

An Empirical Study of Reflective Amplification Traffic Using Honeypots

Distributed Denial-of-Service amplification attacks are a growing threat to modern-day network infrastructure, with the Simple Service Discovery Protocol (SSDP) being one of the most abused protocols for these attacks. This paper revisits the SSDP amplification attacking landscape by designing, implementing and deploying a honeypot system, to capture real-world attack traffic.

The collected data is analyzed to study how adversaries select their amplifiers, what techniques they use and what countries and industry sectors are most often targeted in these attacks.
The results show a clear preference for high amplification reflectors, an increase in the usage of subnet-wide “carpet bombing” strategies, and a concentration of attacks on ISP and hosting infrastructures.

Altogether this research presents an updated overview of the modern-day SSDP DDoS amplification attack landscape. ...
Bachelor thesis (2026) - A. Taneva, Z. Erkin, M.J.G. Olsthoorn
Large language models (LLMs) have shown re- markable performance on mathematical competi- tions (AIME), and recently on the AICrypto bench- mark. AICrypto has tested some of the best commercially-available models on capture-the- flag (CTF)-style cryptography challenges across multiple-choice challenges, proof, and open-ended questions. While they do well on the first 2 cat- egories, the LLMs struggle to solve open-ended questions where advanced mathematical reasoning and creativity are required. This research follows an approach similar to the AICrypto benchmark, testing open-source LLMs and their ability to rea- son. Instead of assigning simple pass/fail scores, the LLM is evaluated qualitatively based on the logic it follows. We aim to demystify the black- box working of commercial LLMs, and potentially lead to developing an open-source framework for solving CTF challenges. The tests are performed on the Qwen3 32B LLM using an agentic ReAct framework. The most common failure modes are discussed, as well their causal factors. ...
Bachelor thesis (2026) - M. Iordache, Z. Erkin, M.J.G. Olsthoorn
Large Language Models demonstrate strong capability in mathematics, yet struggle with cryptographic tasks requiring precise algorithmic implementation. We investigate this capability gap through an ablation study that quantifies the marginal utility of giving the models access to multiple tools. Models interact with these tools through Python REPL with unrestricted access to external libraries to autonomously install and configure their own computational environments within a custom ReAct framework. Using 15 tasks from the AICrypto benchmark covering diverse cryptographic categories, we isolate the contributions of general purpose programming versus domain specific libraries and tools across three configurations and three models, measuring success rate, time to solve, stability, and iteration depth. Our analysis reveals three principal findings: transitioning from pure reasoning to code execution with python and no additional libraries or access to external software yields a +37.78 percentage point performance increase; expanding to unrestricted access produces diminishing returns due to tool management and installation overhead, with effects varying by model type (reasoning / non reasoning); and tool and Python library utility is task-dependent, with large gains for complex algorithmic and computationally intensive challenges but penalties for pattern-based cryptanalysis. These findings establish the existence of a reasoning capability threshold for LLMs below which additional tools consume planning capacity, demonstrating that the Python Standard Library provides optimal performance for most models and cryptographic categories in our selected dataset. This work provides a quantitative basis for integrating large language models with tool usage in cryptographic CTFs.

https://github.com/bogdansys/Research-Project-TU-Delft-CSE3000 ...

Performance, Efficiency, and Failure Modes of ReAct Agents under Context Constraints

Bachelor thesis (2026) - Y.B. Köse, Z. Erkin, M.J.G. Olsthoorn
While Large Language Model (LLM) agents are increasingly capable in specialized domains, the impact of Context Window Constraints on reasoning stability remains under-explored. In this paper we investigate how strictly controlling context size influences agent performance in solving multi-step cryptographic Capture The Flag (CTF) challenges. We adapted an agentic environment where models attempt to solve crypto challenges under four distinct context length constraints (8k–64k), managed dynamically as a tunable hyper parameter. Our results reveal a non-linear performance curve with a clear saturation threshold between 16k and 32k to-kens, beyond which additional context offers negligible benefit. We observe a distinct shift in failure modes: tight constraints lead to context starvation (hallucination), while unconstrained windows allow the accumulation of error traces, where the presence of prior failed attempts biases the agent toward repetitive action loops which in turn inflates its operating costs and decreases operational efficiency. These findings demonstrate that unconstrained context is not only expensive, but leads to regression in the reasoning stability within logic-heavy domains. We conclude that future bench-marks must explicitly distinguish between capability failures and context-induced failures. Further-more, our findings suggest that engineering strategies should prioritize dynamic context management over the reliance on static, maximized windows. ...
Memory access bugs exist in almost every compiled programming language. To solve this, modern programming languages like Rust use complex variable ownership systems that ensure memory safety. These kinds of ownership systems can be used to distinguish between variables that can only be read and ones that can be safely written to as well. These strict ownership rules are, however, limited when using external libraries. While the Foreign Function Interfaces (FFI) used to show what arguments external functions use can be written with these ownership requirements, enforcing them on the side of the external library is not required, potentially resulting in a discrepancy between what is expected on Rust's side and what happens in the external library.

To solve this problem we propose a novel mechanism for detecting memory safety violations across language boundaries. We implemented it in a tool called MiriPBT: a combination of MIRI, a tool that can enforce ownership rules at runtime, MiriLLI, an extension of MIRI that allows ownership rules to be enforced on the other side of the FFI boundary, and Property Based Testing, which allows us to greatly increase the size of the domain we can test. We use Rust's type system to generate inputs for the PBT and use the runtime checks of MIRI and MiriLLI to check if any ownership rules are violated. Finally we present the result of the PBT in a format an average Rust user can easily understand, helping them resolve any FFI related ownership bugs in their code. ...
Master thesis (2025) - M. de Koning, A. Panichella, Pouria Derakhshanfar, M.J.G. Olsthoorn, S.E. Verwer
Effective LLM-based automated program repair (APR) methods can lead to massive cost reductions and have improved significantly in recent times. However, the validity of many APR evaluations as they are conducted at this point is at risk due to data leakage: Prior research has shown that LLMs can memorize solutions to problems if the evaluation benchmark overlaps with the training set, leading to overinflated results.

In this study, we examine the potential of using metamorphic transformations to mitigate the effects of data leakage. For this, we create a variant benchmark for two popular, well-established benchmarks Defects4J and GitBug-Java, and evaluate the APR performance of several LLMs on these benchmarks and their transformed counterparts. In addition, we investigate to what extent our results align with data leakage metrics from other studies.

Our results show that state-of-the-art LLMs for code repair exhibit significant performance degradation (Up to 4.1% for Claude-3.7-Sonnet) on a metamorphically transformed Defecsts4J benchmark. Moreover, we find a significant correlation between our results and the negative log-likelihood as a metric of data leakage. Our results demonstrate the potential of using metamorphic transformations to mitigate the overinflation of evaluation results due to data leakage. We recommend that researchers report results on both original and metamorphically transformed benchmarks in future evaluations.
...

From Theoretical Guarantees to Real-World Deployments

Reliable communication algorithms have existed for a while that assumed either a global authentication model backed by public key infrastructure or peer-to-peer authentication using shared session keys between pairs of neighboring nodes. Real-life networks, however, do not settle for only one or the other. Instead, they are dynamic, heterogeneous, and often composed of a mix of authentication capabilities across different nodes.

Recent work has introduced hybrid models that combine both authenticated links and authenticated processes to better reflect these real-world conditions. These models aim to preserve the strong correctness guarantees of Byzantine-resilient protocols while reducing their communication overhead and improving practical deployability.

This thesis builds on these ideas by introducing DualRC, a reliable broadcast protocol that explicitly supports hybrid authentication environments. It also presents a novel routed version of the protocol designed to shift complexity away from intermediate nodes and toward the sender and receiver, thereby improving scalability and efficiency. Alongside the theoretical contribution, this work includes the first implementation of both variants of DualRC and evaluates their performance across a range of network conditions, trust assumptions, and deployment scenarios.

The goal of this thesis is to demonstrate that reliable communication in partially trusted networks is not only possible but can be efficient, practical, and scalable—provided that protocol design embraces the diversity and complexity of modern distributed systems. ...
One of our first observations working on this study was that all teams used their own form of Agile, borrowing elements from frameworks like Scrum, but ultimately adapting their collaboration methodology to fit their own projects and preferences. This led 2 of the 3 teams to have a mix of both technical and organisational subjects planned for discussion during meetings.

One of the groups went a step further and employed regular 2--7 hours long in-person working sessions (including breaks). The first hour of one of these sessions was recorded and analysed, since it was in preparation for an upcoming TA meeting and consisted predominantly of organisational discussions akin to a daily Scrum meeting. This made it harder for us to interpret their recording, as many of their discussion threads had started before and ended after our observation periods. ...

Investigating the Correlation Between Prior Programming Experience and Task Division

Bachelor thesis (2025) - J.J. Woods, Merel Steenbergen, Efthimia Aivaloglou, M.J.G. Olsthoorn
In this project, the topic of collaboration in computer science education was researched, specifically regarding the correlation between prior programming experience and task distribution in student software development projects. Students selfreported prior programming experience, including their years of study, years of coding experience, their experience rank from 1-10, and their experience compared to their classmates via a survey. A thematic analysis of transcriptions of weekly sprint meetings was then conducted to capture students’ reasoning behind task allocation. Additionally, a quantitative analysis correlated self-reported experience levels with Git metrics including task types (taken from tags), task difficulty (weight), and task reassignment frequency (assignees). While the quantitative analyses found no significant correlation between prior experience and task distribution, the thematic analysis found that students often cited prior experience, in addition to personal interest and time constraints as common reasons for task distribution decisions. These findings suggest that task allocation is affected by multiple factors. While prior experience may have an influence, other factors are likely more impactful. ...
Cyclomatic complexity often serves as a proxy for code quality. While it has been previously studied within education, its relationship with prior programming experience remains unclear. This study analyzed the correlation between different experience metrics, including total years of programming experience, years of experience in industry and in academia, and cyclomatic complexity in student projects. A thematic analysis of task difficulty further contextualized the findings. Results indicated mostly no significant correlations between prior experience and complexity metrics, with only two significant correlations found. While total years of experience and experience in industry showed to be negatively correlated, experience gained in academia showed no consistent findings. The findings suggest that experience alone does not explain complexity scores, indicating that when used as an evaluation method in collaborative projects the results may be influenced by contextual factors such as project settings and task types ...