This thesis investigates the effectiveness and efficiency of embedding-based drift detection in machine learning systems, focusing on synthetic simulations and real-world production data. Through controlled experiments, we compare vector-based and distribution-based metrics regarding sensitivity to drift, memory and runtime cost, and practical utility for early warning of performance degradation. Results from synthetic drift experiments indicate that vector- based metrics respond quickly to small shifts but tend to saturate early, limiting their ability to differentiate between moderate and severe drift. Distribution-based metrics, by contrast, scale more proportionally across the entire drift spectrum, providing more stable and interpretable signals. Memory and runtime profiling show that vector-based methods are consistently more efficient, while distribution-based approaches incur higher costs. A real-world evaluation using eight years of data from a deployed recommendation system confirms the practical value of these findings. Vector metrics consistently provided earlier signals, on average 87 days before performance drops, compared to distribution metrics, which often lagged. However, distribution metrics offered smoother trends and fewer false positives, making them better suited for long-term monitoring. This thesis also explores trade-offs introduced by embedding compression. Principal Component Analysis (PCA) and KLL Sketches were evaluated for reducing computational overhead. PCA preserves the drift signal better in vector metrics, but is more resource-intensive. In contrast, KLL is highly efficient, but sacrifices sensitivity, particularly in vector space.

Computers have become an essential part of modern life. They are used in a wide range of applications, from smartphones and laptops to data centers and supercomputers. However, the increasing usage of computers has led to a rise in energy consumption, which has significant environmental and economic consequences.

The hardware of computing systems has become more energy-efficient over the years. However, software remains a largely untapped area for energy optimization. The key to reducing energy consumption in computing systems lies in understanding the impact of software on the overall energy usage of the system. So, the need for tools and methods to measure software energy consumption is crucial. Those tools and methods should be flexible enough to be used in different computing environments.

This thesis reviews the current state-of-the-art tools for monitoring software energy consumption. After a list of their capacities, we identify the more promising tools and methods and compare their performance during experiments using different types of workloads. The thesis also presents a new tool called Geryon, which brings together the best techniques found in the tools reviewed. Geryon is a flexible and easy-to-use tool that provides multiple estimations at once. Those estimations are computed via simple power models and regression models trained on-the-fly. It also provides a way to plug in an external power model, allowing for more complex power models to be used. The tool is evaluated on the same workloads as the other tools to provide a fair comparison.

The serverless computing trend is steadily picking up steam over the last few years and is challenging the traditional microservices on Kubernetes model which included inefficiencies like idling. The big three cloud providers AWS, GCP and Azure have different opinions on what serverless computing and serverless applications should look like, how they should be designed and what their architecture should be leading to each cloud going their own separate way. Ultimately, this causes vendor lock-in, fragments the serverless cloud technology landscape, paralyzes other research and development efforts while also harming other stakeholders like users, developers and businesses that depend on these cloud services and products. The heavy reliance on often changing proprietary API that is not compatible with alternatives makes stability a serious concern and enables exploitation of users. This thesis proposes MSA - Monadic approach to Serverless Applications to enables developers to write cloud agnostic serverless applications and tackle vendor lock-in. Users can build complex serverless applications without polluting business logic and migrate from one cloud to another by pressing a button. This thesis makes multiple other contributions such as documenting and showing how to reconcile differences in features between the big clouds in various ways for telemetry, metrics, unifying interactions of equivalent services in different clouds, getting rid of tedious boilerplate code in business logic while remaining cloud agnostic among others. Three serverless applications are built with MSA and benchmarked against equivalent applications build with alternatives. Serverless applications built with MSA frequently outperformed in execution time and RAM usage. These experiments also yielded insights on situations in which certain types of serverless applications perform better on a cloud than others. Lastly, usability of MSA and the quality of solution (solving vendor lock-in) received good scores in a user survey.

Industry 4.0 and the Industrial Internet of Things (IIoT) growth will result in an explosion of data generated by connected devices. Adapting 5G and 6G technology could be the leading enabler of the broad possibilities of connecting IIoT devices in masses. However, the edge solution has some disadvantages, such as the loss of resource elasticity compared to cloud solutions. The research questions of this thesis are whether Deep Neural Networks (DNN)-porting can solve the accuracy-performance trade-off of edge computing solutions and how to implement an edge computing system based on open-source container-orchestrated DNN model inference platforms to enable vertical model autoscaling capabilities.
The thesis shows how porting techniques like structured pruning on DNN enable the accuracy performance trade-off in hardware-constrained settings. It generates models with reduced complexity and size while minimally degrading the accuracy. By using these ported models in the proposed inference platform, the thesis demonstrates how an edge computing system can achieve vertical model autoscaling capabilities, enabling efficient use of computational resources. This research focuses on CPU hardware and Real-Time (RT) request scenarios, where the latency Service Level Objective (SLO) combined with current demand are crucial factors. When the resources in an inference system deplete, the latency of individual requests can increase significantly due to queuing. The results show how an orchestrator can make live model version selections based on the model versions and demand. The proposed system increases the maximum possible throughput compared to the state-of-the-art while avoiding creating a queue in the RT scenario and improving system accuracy when CPU resources are available. Additionally, this work proposes a design to implement these benefits in industry-adopted open-source DNN inference platforms.

Data processing systems have become increasingly important in modern computing, as the volume and complexity of data that needs to be analyzed has grown dramatically. Multiple data processing systems have been and are being developed, that are scalable, resilient and performant.

However, despite the advances made in data processing technology, there are still challenges that need to be addressed in order to optimize the performance, energy efficiency as well as the practical- ity of these systems. One such challenge is the need to effectively manage the underlying system’s resources, including the system’s throughput and the amount of work that each operator has to do and to use optimal data-structures that would lead in faster task processing speeds.

To address this challenge, this thesis proposes the implementation of a high-performance data processing system that exposes the underlying system’s metrics to the application level and applys an innovative way for operator communication, by utilizing an efficient thread-safe data structure. By providing underlying system’s metrics to the application’s scheduler, the scheduler can schedule the tasks optimally according to the current system’s state and adjust the system’s resources during run- time. This alleviates the developers from having to fine-tune the system beforehand and allows the system to tackle fluctuating input workload more efficiently.

This thesis will explore the design and implementation of such system, as well as its impact on the performance, energy-efficiency and resiliency of data processing applications. We provide perfor- mance measurements as well as a qualitative comparison of our system compared to other state-of-the art systems, proving our hypotheses.

Concept drift is an unforeseeable change in the underlying data distribution of streaming data, and because of such a change, deployed classifiers over that data show a drop in accuracy. Concept drift detectors are algorithms capable of detecting such a drift, and unsupervised ones detect drift without needing the data’s actual labels, which can be expensive to obtain. This work is concerned with the implementation and evaluation of two existing unsupervised concept drift detectors based on clustering, UCDD and MSSW, by evaluation on both synthetic and real-world data. Our biggest contribution is in making implementations publicly available. By evaluation, we also realise that UCDD detects drift earlier for simple numerical synthetic datasets, MSSW detects drift earlier for more complex synthetic datasets with categorical features, and none seems suitable for real-world datasets.

When deployed in production, machine learning models sometimes lose accuracy over time due to a change in the distribution of the incoming data, which results in the model not reflecting reality any longer. A concept drift is this loss of accuracy over time. Drift detectors are algorithms used to detect such drifts. Drift detectors are important as they allow us to detect when a classification model becomes inaccurate. Some possible uses of drift detectors can even go as far as detecting adversarial attacks on machine learning algorithms. The detectors discussed in this paper are Margin Density drift detectors. Their evaluation is made within an unsupervised context, where we assume no testing labels are available. In real world applications of machine learning models, this might often be the case, as finding labels is costly. Experiments in this paper have found that margin density detectors can be useful tools in detecting the first drift for synthetic data, even though parameter tuning must be done to achieve high accuracy for some datasets. In an unsupervised environment with more than one drift, the drift detectors are unreliable as was seen in experiments involving real world data. With this paper comes an implementation of margin density detectors.

Various techniques have been studied to handle unexpected changes in data streams, a phenomenon called concept drift. When the incoming data is not labeled and the labels are also not obtainable with a reasonable effort, detecting these drifts becomes less trivial. This study evaluates how well two data distribution based label-independent drift detection methods, SyncStream and Statistical Change Detection for Multi-Dimensional Data, detect concept drift. This is done by implementing the algorithms and evaluating them side by side on both synthetic and real-world datasets. The metrics used for synthetic datasets are False Positive Rate and Latency; for real-world datasets, Accuracy is used instead of Latency. The experiments show that both drift detectors perform significantly worse on real-world than on synthetic data.

Large­scale machine learning frameworks can accelerate training of a neural network by per­ forming distributed training on a cluster using multiple GPUs per node and multiple nodes. Because distributed training on a cluster involves many nodes which need to communicate and load and exchange data, a machine learning framework may at certain times during training not fully utilize the available hardware of the system. Various techniques are as­ sessed in their capability to measure the performance of specific parts of the hardware of a cluster. We present ML Board, a tool that measures and visualizes the utilization of the system while training a neural network model using some of the previously assessed tech­niques, and does so without requiring any changes to the used machine learning framework. ML Board can be used to identify straggling nodes, and by subsequently letting the user select different nodes using the Slurm job scheduler, can help to decrease the training time of a ResNet model by between 15 and 45% when using an ImageNet or CIFAR­10 dataset. Furthermore, the energy used by the GPUs can be measured and used to identify and replace GPUs to reduce the total used energy by between 5 to 16%.

Automatically deriving 3D representations of buildings is a challenging problem which is at the base of a wide range of applications. The DE-RISC project aims to generate a 3D model of the entire city of Rotterdam in The Netherlands, enabling many of these applications. Generating a 3D model of a building can be done in a variety of ways, of which only few are robust, scalable and generalizable. Recognition and reconstruction of architectural floor plans is such a scalable method long researched in literature.
Although these methods generalized poorly initially, recent breakthroughs in computer vision have allowed for the application of deep learning based approaches. Recent floor plan processing methods have shown promising results on single-unit floor plans. Single-unit floor plans are floor plans of single apartments of relatively low complexity. In contrast, multi-unit floor plans describe entire buildings, and are thus significantly larger and of higher complexity. Applying single-unit floor plan processing methods to multi-unit floor plans is not trivial, and results in insufficient accuracy. These methods can therefore not be applied to an entire city, limiting the scalability and generalizability.
This thesis proposes a novel multi-scale floor plan recognition and reconstruction method designed to transform floor plans of arbitrary size into their 3D representations. As no multi-unit floor plan datasets exists, a novel floor plan dataset MURF is presented based on multi-unit floor plans from buildings in Rotterdam. MURF considers seven boundary and opening semantic classes, each with distinct physical properties. The recognition part of the method relies on an FCN employing multi-scale skip-connections, an attention mechanism, and a multi-task training objective to reinforce the learning of multi-scale features. The reconstruction part refines predictions from the recognition step by applying post-processing, vectorization, and visualization in Blender.
The proposed method is compared to floor plan processing models from literature and general stateof-the-art segmentation models by a quantitative and qualitative evaluation. Experimental results show that the proposed method is significantly outperforms existing floor plan processing methods, and performs best out of general segmentation models. A case study on the EMC in Rotterdam demonstrates the generalizability of the proposed method.

Thread pools, integrated in programming languages, packages and dependencies are widely used by developers. Thread pools assume they are running alone on the system, which is not always the case. Previous research has shown that adapting thread pool size has been effective under specific conditions. In this research, scaling the thread pool with respect to CPU and network usage is examined. However, simpler metrics can achieve better results. Two solutions are provided for algorithmically scaling the thread pool, both with a different use-case in mind. Next to that an improved way to collect CPU and network metrics is provided, allowing for real-time thread-based measurements. The result of the scaling solution is improved performance, while also offering reduced energy usage. This research shows that when multiple thread pools are running on the same machine, performance and efficiency is improved.

Cloud gaming is a new paradigm that allows users to play games in the cloud and stream them to a thin client. While there is little research about cloud gaming, containerization technologies such as Docker could provide a virtualization alternative to Virtual Machines, as these suffer from overhead and GPU pass-through constraints. With Rapture, we provide an efficient cloud gaming platform based on containerization. Stable GPU multi-tenancy is achieved by resource restrictions such as frame rate and resolution limiting.
A resource-aware best-fit scheduling algorithm accomplishes workload placement in the cloud gaming cluster. The best-fit algorithm outperforms other scheduling algorithms in node utilization and preservation of Quality of Experience. Furthermore, Checkpoint and Restore technologies enable migration, maintaining a high node utilization when down-scaling services. While migration improves efficiency, Quality of Experience is affected.

Anomaly detection has gathered plenty of attention in the previous years. However, there is little evidence of the fact that existing anomaly detection models could show similar performance on different streaming datasets.
Within this study, we research the applicability of existing anomaly detectors to a wide range of univariate streams. We identify main dependent factors with time series that might influence the difference in performances of popular anomaly detection models across different streams, namely, time series features, data drifts, and disorder. We explore the effects that each of the dependent factors has on the performance of selected anomaly detectors. Based on our findings, we propose an adaptive threshold technique that monitors time series disorder. This technique can be integrated into built-in threshold of various anomaly detectors. We show the usability of the proposed method in improving the performance of the models on selected datasets.

Modern systems generate a tremendous amount of data, making manual investigations infeasible, hence requiring automating the process of analysis. However, running automated log analysis pipelines is far from straightforward, due to the changing nature of software ecosystems caused by the constant need to adapt to user requirements. In practice, these are comprised of a series of steps that collectively aim at turning raw logs into actionable insights. The first step is log parsing which aims to abstract away from raw logs toward structured information. Log parsing is paramount, as it influences the performance of all subsequent downstream tasks that rely on its output. Although previous works have investigated the performance of log parsing, given the increase in data heterogeneity witnessed over the past decades, the validity of current estimates is questionable, as there is a lack of understanding of how log parsing methods perform in modern contexts. Consequently, we investigate the field and, in the process, we discover that misleading metrics are adopted, which produce incomplete performance estimates. Furthermore, motivated by an industry use case within the infrastructure of a large international financial institution, we discover that the current log parsing paradigm is not aligned with what is required in practice. Consequently, to address these current limitations, in this work we contribute with the following. We (1) evaluate the field of log parsing, (2) propose a new log parsing paradigm and create a benchmark dataset to facilitate future research, and (3) propose and evaluate a machine learning model that solves log parsing within the new paradigm.

Serverless computing is a relatively recent paradigm that promises fine-grained billing and ease-of-use by abstracting away cloud infrastructure for developers. There is an increasing interest in using the serverless paradigm to execute data analysis tasks. Serverless functions often interact with external services, which can be considered similar to the concept of side-effects in regular programming. Haskell uses monads to isolate side-effects and to structure the composition of functions using side-effects. This thesis explores whether the concept of the monad can be applied in a serverless computing environment, ideally in a way that is flexible with regards to platform. An abstraction of side-effects was developed in the form of a monadic layer that is added to serverless functions. The monadic layer interacts with monads using an interface and exposes the API of the monads to the user. A monadic implementation was also created for a platform-independent function composition mechanism using orchestrator functions. An implementation of a shared state side-effect has also been created as a practical use-case for the monadic layer, and to explore the usability of monads on platforms with more restricted composition frameworks. The implementations are evaluated on performance, expressiveness and usability.

Data compliance is essential for ensuring that organizations do not run afoul of data protection and privacy legislation. Geographically distributed data is an especially relevant topic because of recent developments in cross-border data protection agreements between the United States and the European Union. We introduce Qompliance, a novel system for automated data-centric compliance evaluation in cloud environments. This approach fills a gap in the research for higher-level data-centric compliance systems with a particular focus on geographically distributed data. Its declarative and extensible policy model allows for defining policies that can govern data movements across borders and is intended to be understandable without explicit knowledge of the governed data by employing a tag-based abstraction layer. The particular challenge is to automate data-centric policy compliance on data movements in a maintainable manner. Qompliance analyzes SQL-defined data movements to extract what data is being addressed and combines this information with additional attributes to match policies in a static manner. Policies can decide whether data movements are allowed and specify requirements on the query and the execution that should be enforced. We provide a qualitative comparison between our approach and related work, and we performed a performance analysis that shows that compliance evaluation can be done in seconds for large sets of policies.

Containerization, a lightweight form of virtualization, increasingly became more popular in the last decade. Containers can offer a level of isolation and privacy to the user, which are not always sought after. High performance computing workloads benefit from having a custom container filesystem, but would suffer from any overhead incurred by isolation. This research assesses overhead from Singularity containers for workloads that work on different levels of the memory hierarchy. Different configurations for number of processes and threading are tested for different types of benchmarks. No significant overhead was found which encouraged to test Singularity outside the high performance computing world. Singularity was implemented in Kubernetes, a dynamic container scheduler, by restructuring and rewriting the deprecated Singularity container runtime interface. The re-newed container runtime interface was implemented in Sykube, a local Kubernetes framework for Singularity. A Kafka streaming application was used as benchmark to assess latency in messaging and deployed in a local Docker container network and a local Singularity container network. The results of the local container networks were compared against the performance of the benchmark in Kubernetes. In the local setup Singularity outperformed Docker, while in Kubernetes setup Docker outperformed Singularity.

Mobile networks deal with an increasing portion of the IP Traffic due to the significant growth in the number of mobile devices and the accompanied lifestyle. A large fraction of this IP traffic is spent on duplicate transfers for the same resources. Previous work has shown that a Content Delivery Network (CDN) based on edge-nodes can reduce redundant backhaul traffic by storing popular content closer to the user. It also shows that a reduced user population per node, as in edge-based caching systems, can have a significant negative impact on caching performance. This effect is attributed to a reduced view on global content popularity. In this work we first create and evaluate a simulator for resource requests that is used to evaluate different caching strategies in an edge network. Our simulation confirms the findings of previous work and inspire three different caching strategies: Cooperative-LRU, User Profiles, and Hybrid with Federated. These strategies include mobility information to help alleviate the reduced knowledge on content popularity and help nodes work together more efficiently. Our results show that we are successful in decreasing the impact of the reduced population using the Cooperative LRU strategy and Profiles strategy. We then improve upon that performance by using a Hybrid strategy of Federated nodes and one of the mobility strategies.

With the advent of the cloud-native paradigm, the software development and deployment style has significantly reformed. An increasing number of enterprises are migrating their microservice applications onto Kubernetes, a production-grade container orchestration platform, to fully use the advantages of Kubernetes in self-healing, service discovery, auto-scaling, etc. Kubernetes introduces a host of novel software design ideas but also poses challenges to newcomers. In this thesis, we focus on the cold start problem in microservice auto-scaling on Kubernetes. A cold start refers to a situation where services receive overwhelming requests from the outside but has no sufficient backend containers to serve. As the auto-scaling process is launching slow, the cold start problem may elongate response latency or even cause service unavailable in a more severe case. To study and mitigate the adverse effect of cold start, we pose three research questions.

The first research question is to identify what factors contribute to the cold start. We first learn lessons from research on cold start in Serverless and examine the current service auto-scaling process powered by Horizontal Pod Autoscaler (HPA), then suggest five hypothetical factors that may affect cold-start performance. Our findings indicate that reactive autoscalers like HPA are dilatory in determining scale events and lack a coordination mechanism to scale correlated workloads in sync. Besides, different programming languages differ in code loading and execution time. The major factors lie in slow scale decision making and application code loading. In contrast, other factors such as cluster size, the choice of container runtimes and CNI network solutions are negligible.

In the next two research questions, we investigate how to mitigate cold start and how effective the proposed solutions are. Based on our study from the first research question, we conclude that extra efforts must be made to speed up scaling decision-making and reduce application startup time. We compare a few technical solutions to the hypothetical factors. According to the results, we encourage microservice application developers, especially Java developers, to try and test different programming language stacks, preferably Go, for rapid auto-scaling. In addition, we propose a coordinated horizontal pod autoscaler, named CHPA, a supplement to HPA, allowing a group of identified, correlated scale targets to scale out in sync, rather than each making independent scaling decisions. As a result, no laggard pod in the auto-scaling process, thereby mitigating the cold start problem holistically.

Serverless computing is an emerging paradigm for structuring applications in such a way that they can benefit from on-demand computing resources and achieve horizontal scalability. As such, it is an ideal substrate for the resource-intensive and often ad-hoc task of training deep learning models. However, the design and stateless nature of serverless platforms make it difficult to translate distributed machine learning systems directly to this new world. With KubeML, we present a purpose-built serverless machine learning system that runs atop Kubernetes and seamlessly embeds into the popular PyTorch framework. Unlike alternative systems, KubeML fully embraces GPU acceleration and is able to outperform TensorFlow, especially with smaller local batches, while allowing for higher resource density. KubeML reaches a 3.98x faster time-to-accuracy with small batch sizes, and maintains a 2.02x speedup between the top results of both platforms for commonly benchmarked machine learning models like ResNet34.