The advancement of data systems demands continuous learning, yet traditional educational materials often fall short of meeting evolving learning needs. YouTube has emerged as a widely used platform for informal learning, but its role in data systems education remains underexamined. This thesis addresses that gap by constructing a curated dataset of 17,434 instructional YouTube videos related to data systems, focusing on content availability and organization, key video characteristics, factors influencing audience engagement, and subtopic coverage in SQL education. Using a curriculum-aligned query strategy and a machine learning filtering pipeline, the dataset maintains relevance to educational objectives and offers broad topical coverage within the data systems domain. Video characteristics are analyzed across dimensions such as content volume, engagement metrics, transcript availability, language, geographic origin, and topic distribution. Findings reveal that content and engagement are highly uneven, with a small subset of videos, channels, languages, countries, and topics capturing disproportionate attention. Statistical modeling shows that engagement in this domain is positively associated with longer video duration, SQL focus, and high-subscriber channels, while overly long titles, frequent uploads, and older channels correlate negatively. Subtle patterns suggest that culturally or regionally tailored content may further enhance engagement. While SQL-related topics dominate in volume and engagement, a subtopic classification of 4,242 SQL videos reveals that although 87\% of textbook-derived subtopics are covered, content is heavily concentrated on core querying and schema commands. Advanced, theoretical, systems-level, and integration-related topics are rarely addressed.

Frequent route changes in modern SDN-based net works are known to severely degrade the performance of TCP Cubic. This degradation is caused by two factors: sudden RTT changes, and packet reordering which Cubic misinterprets as congestion. This research investigates how a modern alter native, BBRv3, performs under these same conditions. Using ns-3 simulations with rerouting intervals of 3, 5, and 10seconds, we show that BBRv3is significantly more resilient. While Cubic through put is reduced by nearly 50% at 3-second intervals, BBRv3 performance degrades by less than 10%, as its probing mechanism does not use packet re ordering as a congestion signal. We also examined a second scenario where a flow leaves a saturated link. We concluded Cubic flows take much longer to fill the newly freed up bandwidth, as much as 8 seconds for a 25% increase in available bandwidth. BBR performs much better as its able to recognize that the link is not saturated during its next probing phase and mediately fill it. Therefore, we conclude that BBRv3 is better suited for the dynamic network conditions found in SDN environments

The Transmission Control Protocol (TCP) remains the cornerstone of modern network communication, enabling reliable and ordered data delivery across a wide range of network environments. Despite its ubiquity, TCP’s variants’ performance under extreme and highly variable network conditions remains insufficiently explored. This paper investigates the behavior of two common TCP variants - CUBIC and BBRv1 - when subjected to dynamic bandwidth and delay fluctuation. Such conditions are increasingly common in real-world wireless networks and can have a significant impact on TCP flows. We conduct a series of tests using the ns-3 framework, employing a dumbbell topology to simulate wireless connections. The results of 6 different testing scenarios are presented. They showcase that both algorithms experience significant packet loss in the event of bandwidth variance in-transmission, with BBRv1 adapting to these changes better, but not dominating over CUBIC in a multi-flow connection. In addition, both TCP variants experience harsh throughput drops and lose very few packets in the event of delay spikes. When faced with both delay and bandwidth variance, BBRv1 experiences high packet loss while CUBIC’s connection remains stable.

Modern TCP congestion control algorithms rely on timely ACK feedback to adjust their parameters. However, some networks deliberately suppress ACKs. This study uses the ns-3 simulator to experiment with the impact of suppressing ACKs on the reverse path on four TCP variants (BBRv3, Cubic, NewReno, Vegas) in both wired and wireless dumbbell topologies. In wired experiments, we implement a custom queue that allows us to test fixed aggregation ratios at the transport-layer. In wireless scenarios, we utilize the IEEE 802.11n standard’s MAC-layer aggregation schemes (A-MSDU, A-MPDU), both individually and in combination, and also evaluate varying maximum A-MPDU aggregation sizes. Our results show that while transport-layer aggregation can degrade performance and fairness, especially for BBRv3 and Cubic, MAC-layer aggregation consistently improves throughput without destabilizing TCP behaviour. NewReno demonstrates strong resilience across both setups, while Vegas exhibits highly inconsistent performance. These findings highlight the importance of aligning aggregation
mechanisms with congestion control strategies carefully.

The Low-Latency, Low-Loss, ScalableThroughput (L4S) service aims to support real-time applications by enabling high throughput with sub-millisecond queueing delay. It combines
scalable ECN-based congestion control (e.g., TCP Prague) with a Dual-Queue AQM such as DualPI2 to separate low-latency and classic traffic.
This paper evaluates how L4S behaves under stressful network conditions using an extended ns-3
testbed with DualPI2, TCP Prague, and ECNenabled BBRv3. We test five scenarios: RTT jitter, bandwidth shifts, mixed traffic, wireless loss, and scalable-to-scalable coexistence.
Our results show that TCP Prague consistently delivers low delay and stable throughput, whereas
legacy TCP Cubic shows elevated and more variable delay—especially under jitter and in shared
queues. ECN-BBRv3 coexists cleanly with Prague, but when Cubic and Prague share a queue, Prague dominates bandwidth. L4S thus meets its latency goals, but fairness with classic TCP remains an open issue.

Most TCP data transfers in the Internet are short. This makes the startup algorithms an important factor that impacts TCP performance. Several startup algorithms have been developed. However, not a lot of research has been conducted into how these behave and interact when used for short flows. This paper aims to provide a thorough evaluation of these algorithms and their interactions under different network conditions, focusing on short flows and using ns-3. We have observed that JumpStart seems to outperform the other algorithms used when it comes to flow completion time for short flows. That is likely because it starts to send data with an aggressive initial congestion window and the flow is finished after the first few RTTs. However, JumpStart performs more poorly when the flows are longer. We have shown that JumpStart has a great potential to make communication more efficient in the Internet but further research has to be conducted into its behavior in more adversarial conditions that represent real life situations better.

In today’s rapidly evolving software landscape, where continuous integration and continuous delivery are paramount, the presence of flaky tests poses a significant obstacle. These tests, exhibiting unpredictable pass/fail behavior, hinder development progress, waste valuable resources, and erode developer trust. This research delves into the root causes and mitigation strategies for flaky tests within a large-scale, database-driven industrial setting: Exact.

The increasing reliance on databases in modern software systems, including Exact’s own platform, necessitates a deeper understanding of the unique challenges posed by database-dependent tests. By analyzing flaky test behavior through repeated test runs on the same code, we identified key contributors to flakiness, including resource contention, test order dependencies, ‘dirty tests’ that leave the system in an inconsistent state, platform-specific issues, and combinations thereof.

Based on the root causes for flakiness at Exact, we developed and evaluated three mitigation strategies and supporting tools: minimizing redundant database background tasks, explicitly disposing of test data, and disabling database dirty tests. Our study resulted in a substantial reduction in flakiness, leading to a significant increase in the release rate from Exact from 60% to 96%. We improved the chance of their CI/CD pipeline passing with no code changes from 27% to 95%.

Furthermore, this research highlights the importance of collecting and analyzing rich, granular test data to identify patterns and root causes of flakiness. Providing developers with actionable information from this analysis motivates them to address flakiness proactively. Moreover, understanding the interplay between different types of tests, such as the impact of dirty tests on other seemingly unrelated tests or in combination with other factors, is crucial for effectively mitigating cascading failures.

WebDSL is a DSL for creating web applications, combining many different aspects and domains of web design in a single language. The dynamic semantics of this language are not defined, despite multiple attempts, abandoned due to complexity of the language and lack of expression of chosen frameworks. We adapt the algebraic effects and handlers approach and the framework introduced in Datatypes a la carte (Swierstra, 2008) to create a modular denotational semantics model of WebDSL, extending the framework by a bifunctor formulation for multi-sort syntax definition that allows us to distinguish between effects raised by different components of the language. In the process of defining the framework and semantics in Haskell, we encountered obstacles and of working with algebraic effects and handlers paradigm in the language, leading us to compile workarounds, solutions and pitfalls to avoid when constructing and maintaining such model. In evaluation of the framework approach with earlier attempts at defining dynamic semantics for WebDSL, we find algebraic effects and handlers to be a viable and successful approach for modelling a rich DSL such as WebDSL, and propose possible improvements to the WebDSL compiler.

Automated software testing is a frequently studied topic in specialized literature. Search-based software testing tools, like EvoSuite, can generate test suites using genetic algorithms without the developer’s input. Large Language Models (LLMs) have recently attracted significant attention in the software engineering domain for their potential to automate test generation. UTGen, a tool integrating LLMs with EvoSuite, produces more understandable tests than EvoSuite; however, the generated tests suffer a coverage drop.

To streamline bug detection by developers, we propose UTGenCov, a concept that focuses on improving the understandability of EvoSuite-generated tests without compromising on coverage. This approach builds upon UTGen by thoroughly analyzing the reasons behind the decrease in coverage and proposing an alternative approach.

Our investigation determined that the leading cause of coverage reduction in UTGen is LLM hallucination in the Understandability phase. UTGenCov aims to address hallucinations by providing the source code of the methods used in the test to the LLM. Yet, our experiment results indicate inconsistent performance and a further decrease in branch coverage of 0.74% compared to UTGen.

Writing tests enhances quality, yet developers often deprioritize writing tests. Existing tools for automatic test generation face challenges in test under- standability. This is primarily due to the fact that these tools fail to consider the context, leading to the generation of identifiers, test names, and identifier data that are not contextually appropriate for the code they are testing. Current metrics for judging the understandability of unit tests are limited as they do not take into account contextual factors such as the quality of comments. Developing a metric to evaluate test readability is essential for selecting the most comprehensible tests. This research builds on UTGen, incorporating LLMs to enhance the readability of automatically generated unit tests. We developed a readability score and used LLMs to evaluate and rank tests, comparing these rankings with human evaluations. This research concludes that LLMs can successfully evaluate the readability of test cases. The GPT-4 Turbo Simple Prompt model exhibited the best performance, with a correlation of 0.7632 with human evaluations. Through comparing different LLMs and techniques for as- signing readability scores, we identified approaches that closely matched human evaluations, demonstrating that LLMs can successfully rate the read- ability of test cases.

Automated test generation is the means to produce correct and usable code while maintaining an efficient and effective development process. UTGen is a tool that utilizes a Large Language Model (LLM) to improve the understandability of a test suite generated by a Search-Based Software Testing tool, namely EvoSuite. Often while the LLM attempts to improve a given test case, it generates code that is too far from the original, changing the test's purpose. Alternatively, it may generate code that does not compile. Such behaviour is called ``LLM Hallucination".

The current hallucination handling of UTGen is time-consuming and resource-expensive. To address this, we propose two alternative approaches that use information retrieval prompt engineering techniques to minimise hallucinations. Our respective techniques include incorporating the source code under test and the errors thrown by the latest generated test case to the LLM prompt. We assess our methods through a comparison study against the base UTGen version. We observe that source code retrieval enhances the generation of compilable test cases for complex classes. Error code retrieval shows similar hallucination performance to base UTGen, with a decrease in the number of re-prompts for classes with a high normalised Lack of Cohesion of Methods (*LCOM).

Index Terms - Automated Test Generation, Large Language Models (LLMs), LLM Hallucination, Prompt Engineering

Since software testing is crucial, there has been research on generating test cases automatically. The problem is that the generated test cases can be hard to understand. Multiple factors play a role in understandability and one of them is test summarization, which provides an overview of the test of what it is exactly testing and sometimes highlights the key functionalities. There already exist some tools to generate test summaries that use template-based summarization techniques. Limitations of generated summaries include that they can be lengthy and redundant, and that it is best to use them in combination with well-defined test names and variables. There is a tool developed named UTGen, which combines Evosuite and Large Language models to increase understandability which includes improving the test names and variables, but does not have a summarization functionality yet. In this research, we extend UTGen using LLM-generated summaries. We investigate to what extent the understandability of a test case can be influenced by LLM-generated test summaries in terms of context, conciseness, and naturalness. For this reason, we do a user evaluation with 11 participants with a software testing background. They will judge LLM-generated summaries and compare them to existing summarization tools. The LLM-generated summaries scored overall higher than the template-based summaries and were also more favorable by the participants.

Automated software testing plays a critical role in improving software quality and reducing manual testing expenses. However, generating understandable and meaningful unit tests remains challenging, especially with frameworks optimized for coverage like Search-Based Software Testing (SBST). Large Language Models (LLMs) have the capability to generate human-like text, while capture/replay techniques can provide realistic data scenarios through trace logs, contributing to meaningful test case generation. This study introduces UTGen+, an approach that enhances LLM-based SBST by integrating trace logs from end-to-end tests, aiming to further improve test case understandability.
We conducted a comparative user study with 9 participants using UTGen+, original UTGen, and conventional SBST (EvoSuite), focusing on the effects of trace log inclusion on the naturalness and relevancy of comments, identifiers, and test data across several projects. The results indicated that while UTGen+ did not improve the naturalness and relevancy of comments and identifiers, it significantly enhanced the relevancy of test data. These findings suggest that incorporating contextual data can indeed benefit the generation of more relevant and understandable automated test cases.

The advancement of Machine Learning (ML) in the last decade has created new business prospects for developers working on ML models. Models that are expensive and time-consuming to design and train can now be outsourced from others to reduce costs using Machine Learning as a service (MLaaS). \textbf{Deep Neural Networks (DNNs)} are particularly expensive to train, therefore many who need a DNN utilize the services of an MLaaS provider. This creates the \textbf{possibility of piracy} of this valuable asset, and the need to prevent piracy to assure a fair market. To address this need, research has been conducted on protecting DNNs using various watermarking techniques. A work by \textit{Li et al.} has proposed null-embedding, a technique that renders the DNN useless if it is subject to a piracy attack. Despite being effective, this method was shown to reduce classification performance when embedding a watermark into the model. This paper suggests modifications to the null-embedding technique that reduce this impact and keep the classification accuracy close to that of a non-watermarked model.

Data security has become more important over the last few years as data sharing over the world has become trivial. Data ownership therefore becomes critical as data can be very valuable and vulnerable to theft. Watermarking is a technique that can help data owners prove ownership over their data. In this paper, an approach is presented to watermark data that is gathered over time, such as weather data. With this research, we propose an adaptation of an existing audio watermarking technique developed in [1]. The adapted algorithm embeds a bit stream into a non-medical time series dataset by calculating the Discrete Wavelet Transform coefficients and modifying their magnitudes. The algorithm shows good robustness against a small range of data modification attacks but lacks capability in larger
scaled attacks. In addition, the proposed algorithm does require additional research to use it in a professional setting.

AI and machine learning have been topics of big interest in the last couple of years, with plenty of applications in many domains. To train these models into useful and desirable tools, a large amount of data is necessary. This data is expensive to collect, becoming one of the most valuable commodities of this century. As the value of data increases, protecting this intellectual property becomes more and more relevant. Watermarking is a technique widely used for data protection in media, but the non-media counterpart has not been researched as thoroughly. In this paper, an adaptation of a common watermarking technique, DWT watermarking, is applied on two datasets used for machine learning. This technique is invisible and robust in signal watermarking, but its performance on a numerical dataset has not been previously researched. A previously devised algorithm was used, but it was adjusted to better fit dataset watermarking. To assess the quality of the watermark, the marked data has been subjected to create, remove, update and zero-out attacks. On top of this, multiple machine-learning models have been trained on the marked data. Initial results show that the proposed technique performs well in terms of invisibility, obtaining similar or better accuracies than models trained on the original data, but it is quite sensitive to attacks. Even small modifications, less than 1\% of the data, can break the signature.

In the realm of machine learning (ML), the need for efficiency in training processes is paramount. The conventional first step in an ML workflow involves collecting data from various sources and merging them into a single table, a process known as materialization, which can introduce inefficiencies caused by redundant data. Factorized ML strives to reduce this by maintaining the original data forms and performing model training on the separate source tables. This approach can lead to significant increases in training efficiency.

However, factorized training does not always reduce cost compared to traditional materialized training. This research tackles this issue by examining the multidimensional cost optimization problem that emerges when deciding between factorized and traditional materialized learning methods. It fills in gaps left by prior research, which is focused on CPU-based training, by investigating the cost estimation landscape for factorized ML, with a special emphasis on GPU performance compared to CPUs. The used factorized ML framework is expanded to incorporate GPU training, a topic not explored in previous research. We demonstrate that GPU training exhibits significantly different cost characteristics than CPU training, which has substantial implications for the design of cost models and the optimization of factorized ML.

Through an empirical study, an ML-based cost model is developed that can accurately predict the faster training method for a wide range of scenarios. On an extensive evaluation with real-world datasets this model boasts an average speedup of 3.8x, versus the state-of-the-art's 0.9x. We also show that it generalizes to scenarios with datasets and hardware settings on which the model is not trained, keeping 82% of training set performance.

Our innovative cost model for factorized ML enables significant time savings in training-intensive scenarios and further underlines the benefits of factorized training. However, effort should be invested into incorporating factorized training into existing ML frameworks so this method of training a model, and our cost model, can be evaluated in a larger set of realistic scenarios.

Large language models (LMs) are increasingly used in critical tasks, making it important that these models can be trusted. The confidence an LM assigns to its prediction is often used to indicate how much trust can be placed in that prediction. However, a high confidence can be incorrectly trusted if it turns out to be incorrect, also known as a high-confident error, or unknown unknown (UU). Blind spots, clusters of UUs, are caused by out-of-distribution data (OOD), bias, covariate shift, or unseen data. Previous work by Lippmann et al. generated samples based on these UUs, however, they selected a random subset of all found UUs. Although this was the first study that could mitigate blind spots without the need for crowd workers or oracles, they did not address the underlying causes of blind spots, such as OOD and bias. Data valuation is the research field that is concerned with valuing samples to improve model performance and can identify OOD data, bias, and noise. This paper proposed to combine the research field of data valuation, with the mitigation of blind spots. We generated synthetic samples using the highest-valued samples and retrained the LM using a Weighted Loss based on Data Values (WLDV). We conducted an extensive evaluation of our approach on four tasks, demonstrating a reduction of UUs by up to 32.7%, while retaining the same level of accuracy. This was the first exploration of combining the research fields of data valuation and blind spot mitigation.

Online gaming is the world’s largest entertainment industry by revenue, and supports over 3 billion consumers worldwide. Many of the world’s most popular online games must manage millions of concurrent players through a single unified service. Achieving performant and scalable online games is challenging. Online games are subject to stringent quality of service requirements, notably extremely low response times, with at most 50ms being considered acceptable. Unlike many other types of applications, the performance of online games depends to a large degree on the resources available on end-user devices. These devices are typically heterogeneous, limited in compute and network resources, and subject to unpredictable changes in resource availability.
Addressing this challenge, we propose in this work the concept of differentiated deployment, which allows online games to selectively manage and scale online-game systems with fine granularity in response to changes in available resources. We design Polka, a framework for online games which supports differentiated deployment. We then implement PolkaDOTS, an open-source proof of concept of the Polka framework built in an industry standard game development ecosystem.
We evaluate our approach using Dither, a custom-built experiment runner for large scale distributed experiments on online games. We use Dither to perform real-world experiments on a representative Minecraft-like Game, Opencraft 2, built on the PolkaDOTS stack, and analyze the impact of various differentiated deployment scenarios. From these experiments, we find that differentiated deployment can decrease performance variability of online-game servers, and decrease the response time experienced by players by up to 32%. Most importantly, we show that differentiated deployment enables novel deployment techniques, including switching from local rendering to cloud-based rendering (i.e., cloud gaming) at runtime.

The current trend towards the integration of artificial intelligence (AI) and graphics processing unit (GPU) technologies has resulted in the development of embedded hybrid GPU-AI accelerators, which offer high computational power and energy efficiency. One of the key challenges in designing such accelerators is ensuring their timing correctness, as any timing violation may lead to system failure and incorrect results. To address this challenge, timing simulators have been proposed as a promising solution, as they enable accurate and efficient timing analysis of these complex systems. Nevertheless, such simulators have their limitations: detailed ones, such as cycle-accurate simulators, have high accuracy but exhibit high overhead, while faster approaches, like event-driven simulators, usually cannot achieve high accuracy levels. Therefore, the question arises: How can we implement a timing simulator to achieve a balanced trade-off between accuracy and execution time?

In this context, we introduce NEOX-V, a cutting-edge RISC-V based GPU Processor optimized for GPGPU and AI workloads. However, the current NEOX-V product lacks timing information in its simulator. This has prompted us to use it as a case study to bridge the gap between accuracy and execution time in timing simulators. This is achieved through the implementation of a new feature: a timing simulator that utilizes event-driven modeling.

To assess the proposed simulator's accuracy and effectiveness, we employ a comprehensive validation framework, using diverse workloads and configurations, from simple micro-benchmarks to intricate AI tests. The results demonstrate that the timing simulator achieves an accuracy error below 8\% when compared to the RTL equivalent for all applications, with a marginal increase in actual simulation time of only 0.7\%. It is worth noting that the timing simulator's utility extends beyond predicting execution time; it also plays a crucial role in verifying the existing design and uncovering its limitations.

Overall, this thesis makes a significant contribution to the field of computer architecture by providing a powerful tool for the design, development, and evaluation of an embedded hybrid GPU-AI accelerator called NEOX-V. It is our hope that this work will inspire further research and development in this exciting and rapidly evolving field.