Online gaming is the world’s largest entertainment industry by revenue, and supports over 3 billion consumers worldwide. Many of the world’s most popular online games must manage millions of concurrent players through a single unified service. Achieving performant and scalable online games is challenging. Online games are subject to stringent quality of service requirements, notably extremely low response times, with at most 50ms being considered acceptable. Unlike many other types of applications, the performance of online games depends to a large degree on the resources available on end-user devices. These devices are typically heterogeneous, limited in compute and network resources, and subject to unpredictable changes in resource availability.
Addressing this challenge, we propose in this work the concept of differentiated deployment, which allows online games to selectively manage and scale online-game systems with fine granularity in response to changes in available resources. We design Polka, a framework for online games which supports differentiated deployment. We then implement PolkaDOTS, an open-source proof of concept of the Polka framework built in an industry standard game development ecosystem.
We evaluate our approach using Dither, a custom-built experiment runner for large scale distributed experiments on online games. We use Dither to perform real-world experiments on a representative Minecraft-like Game, Opencraft 2, built on the PolkaDOTS stack, and analyze the impact of various differentiated deployment scenarios. From these experiments, we find that differentiated deployment can decrease performance variability of online-game servers, and decrease the response time experienced by players by up to 32%. Most importantly, we show that differentiated deployment enables novel deployment techniques, including switching from local rendering to cloud-based rendering (i.e., cloud gaming) at runtime.

The ongoing digitalization of the world, estimated to reach a yearly data generation of 200 Zettabytes by 2025, is putting increasing pressure on system developers to provide systems capable of scaling with future needs. Of particular importance are the data storage systems, providing the means of storing and retrieving the vast amounts of data. One widely adopted storage technology, predicted to become the leading media for future data storage, is flash-based solid state drives (SSD). The complex architecture of flash SSD however introduces several challenges, such as necessary garbage collection, for managing the flash storage. To readily integrate flash SSD into storage systems, the flash management idiosyncrasies are hidden inside the storage device. The hiding of the flash management idiosyncrasies has however been identified to have significant performance implications. As a result, numerous efforts have pushed towards more open flash storage interfaces, with the most recent addition of Zoned Namespace (ZNS) flash SSD. ZNS SSD presents a unique opportunity for storage software and the flash SSD to coordinate the flash management responsibilities.

While the open flash storage interface of ZNS presents a plethora of opportunity in software optimization, current software support is in its early stages, leaving much of its potentials yet to be explored. In this work, we present msF2FS (multi-streamed F2FS), a file system with optimized ZNS integration, based on the de facto standard flash file system F2FS. msF2FS enhances the ZNS integration by leveraging the parallelism capabilities of ZNS SSD, and increasing the coordination between the file system and applications for data placement decision-making. The data placement coordination between file system and application reduces the sub-optimal data placement decisions made by the file system. Evaluations of msF2Fs show the benefit of the application and file system coordination, with the RocksDB application achieving up to 23.19% higher throughput as a result of optimized data placement. We make all developed code of msF2FS publicly available at https://github.com/nicktehrany/msF2FS.

Graphs are versatile enough to model many problems, and we see them used for
solving a large spectrum of challenges in various disciplines. As the size of graph
data grew over the years, to satisfy the requirement to process large data sets on
time, distributed graph-processing systems were implemented.
However, these systems were built with a static deployment in mind, only allowing the systems to run on a pre-defined set of machines. Manual optimizations
and tweaks to deployment configurations lead to a waste of resources as well as an
increase in operational costs.
In this thesis, we explore the introduction of elastic deployment in distributed
graph-processing systems. We design and implement a distributed graph-processing
system and a framework for assessing elasticity and performing real-world experimentation.

Cloud datacenters underpin our increasingly digital society, serving stakeholders across industry, government, and academia. These stakeholders have come to expect reliable operation and high quality of service, yet demand low cost, high scalability, and corporate (environmental) responsibility. Datacenter operators are confronted frequently with highly complex decisions that involve numerous aspects of risk. The consequence of bad decisions can be financial penalties or even loss of customers on the one hand, or a competitive disadvantage or unsustainable environmental impact on the other hand. Despite risk analysis being an integral part of the design and operation of cloud infrastructure, relatively few comprehensive approaches and tools exist, leaving many datacenter operators ill-equipped to make informed decisions with confidence.

We propose Radice, an instrument for data-driven analysis of IT-related operational risks in sustainable cloud datacenters. Unlike most state-of-the-art approaches used by the industry, Radice automates the process of risk analysis in datacenters and utilizes the large and diverse volume of data reported by the monitoring systems in datacenters, including environmental data. Underpinning this system is the trace-based, discrete-event simulator OpenDC, which enables the exploration of many risk scenarios through its support for diverse workloads, datacenter topologies, and operational phenomena. Radice’s interactive and explorative user interface assists datacenter operators in addressing complex decisions involving risks, providing them with actionable insights, automated visualizations, and suggestions to reduce risk.

We implement Radice and conduct a comprehensive evaluation of the system to demonstrate how it can aid datacenter operators when confronted with fundamental risk trade-offs. Although Radice is designed to work across many kinds of datacenters, in this work, we focus on private-cloud, business-critical workloads, and on public-cloud operations, representing the majority of workloads in Dutch datacenters. Our experiments show many interesting findings, supporting our claim for a need for data-driven risk analysis in datacenters. We highlight the increasing risk faced by datacenter operators due to price surges in the electricity and CO2 bond markets, and demonstrate how Radice can be used to control such risks. We further show that Radice can automatically optimize topology and operational settings in datacenters for risk, revealing configurations that reduce the overall risk by 10%–30%. Following extensive performance engineering, Radice is able to evaluate risk scenarios by a factor 70x–330x faster than others, opening possibilities for interactive risk exploration. We release Radice as free and open-source software for the community to inspect and re-use.

Cloud datacenters provide a backbone to our digital society. Crucial to meeting increasing demand while maintaining efficient operation is the activity of capacity planning. Inaccurate capacity planning for cloud datacenters can lead to significant performance degradation, denser targets for failure, and unsustainable energy consumption. Although this activity is core to improving cloud infrastructure, relatively few comprehensive approaches and support tools exist, leaving many planners with merely rule-of-thumb judgement.

We propose Capelin, a data-driven, scenario-based capacity planning system for cloud datacenters. We design Capelin to address requirements we have derived from a unique survey of experts in charge of diverse datacenters in several countries. Capelin introduces the notion of portfolios of scenarios, which it leverages in its probing for alternative capacity-plans. At the core of the system, a trace-based, discrete-event simulator enables the exploration of different possible topologies, with support for scaling the volume, variety, and velocity of resources, and for horizontal (scale-out) and vertical (scale-up) scaling. The approach centers around a notion of portfolios of scenarios as a framework for probing alternative decisions and courses of events. Capelin gives detailed quantitative operational information for each scenario, which could facilitate human decisions in capacity planning.

We implement and open-source Capelin, and show through comprehensive trace-based experiments it can aid practitioners. Although Capelin is designed to work across many kinds of datacenters, in this work we focus on private-cloud, business-critical workloads, and on public-cloud operations. The results give evidence that choices that seem reasonable and common in practice could be worse by a factor of 1.5-2.0 than the best, in terms of performance degradation or energy consumption. We also show evidence of Capelin identifying meaningful choices that are different from the baseline proposed by a team of professional datacenter engineers. We open-source Capelin and release data artifacts for public inspection and reuse.

Datacenter infrastructure has become vital for stakeholders across industry, academia and government. To operate efficiently, datacenter operators rely on a variety of complex scheduling techniques, to distribute user workloads across resources. In this work, we leverage a reference architecture for datacenter scheduling to design and implement an instrument for systematic design space exploration of datacenter schedulers. We construct a formal representation of the design space for datacenter schedulers, using scheduling policies collected from real-world schedulers. We then use a genetic algorithm in combination with trace-based simulation to explore the space, optimizing for workload metrics. Through several experiments, we assess the viability of the instrument. We find that our instrument is able to identify patterns in the workloads and adapt the scheduling policies appropriately. Overall, our work leads to numerous findings, which can become valuable for future comprehension and development of schedulers.

The need for accessible and cost-effective IT resources has led to the
near-universal adoption of cloud computing. Within cloud computing,
serverless computing has emerged as a model that further abstracts away
operational complexity of heterogeneous cloud resources. Central to this form of
computing is Function-as-a-Service (FaaS); a cloud model that enables
users to express applications as functions, further decoupling the application
logic from the hardware and other operational concerns. Although FaaS has seen
rapid adoption for simple use cases, there are several issues that impede its
use for more complex use cases. A key issue is the lack of systems that
facilitate the reuse of existing functions to create more complex, composed
functions. Current approaches for serverless function composition are either
proprietary, resource inefficient, unreliable, or do not scale. To address this
issue, we propose an approach to orchestrate composed functions using reliably
and efficiently with workflows. As a prototype, we design and implement
Fission Workflows: an open-source serverless workflow system which leverages the
characteristics of serverless functions to improve the (re)usability,
performance, and reliability of function compositions. We evaluate our prototype
using both synthetic and real-world experiments, which show that the system is
comparable with or better than state-of-the-art workflow systems, while costing
significantly less. Based on the experimental evaluation and the industry
interest in the Fission Workflows product, we believe that serverless workflow
orchestration will enable the use of serverless applications for more complex
use cases.

MapReduce ecosystems are (still) widely popular for big data processing in data centers. To address the diverse non-functional requirements arising from many and increasingly more sophisticated users, the community has developed many scheduling policies for MapReduce workloads. Although some individual policies can dynamically optimize for single and stable performance objectives, such as minimizing runtime or cost, or meeting deadlines for realtime-jobs, it seems unlikely that individual policies will remain competitive for increasingly more dynamic workloads and objectives. In contrast, in this work we investigate the ability to dynamically balance performance and cost of a portfolio scheduler for MapReduce workloads. To this end, we design and implement a portfolio scheduling technique, that is, a system capable of adapting to the current workload characteristics and target objectives by periodically evaluating its set of potential policies, and of switching to "the best" policy that targets the current system state. We implement and evaluate our system with real-world experiments on a workload containing a mixture of real-time and batch jobs, with the purpose of minimizing deadline violations, while keeping batch job slowdown in check. Our results show that POSUM is a promising alternative: it can predict map task runtimes accurately when calculating average input processing rates, while reduces need a more complex model that accounts for an application-dependent component and variability. However, even without precise predictions, the proposed system can out-perform the individual policies of its portfolio for the combined optimization goal.

Big data, the large-scale collection and analysis of data, has become ubiquitous in the modern, digital society. Within the big data landscape, graphs are widely used to study collections of entities and the complex relationships that connect them. The analysis of graphs has applications in social networks, logistics, finance, bioinformatics, and many other domains. With the rapidly increasing amounts of data being collected, analyzing large-scale graphs has become a necessity. To address this need, many dedicated graph analysis frameworks have been developed in recent years. However, their performance is poorly understood. In this thesis, our goal is to improve insight into the performance of graph analysis frameworks. We design the Graphalytics ecosystem, a set of complementary systems for understanding the performance of graph analysis frameworks, with a focus on two key components. First, we design, implement, and evaluate Graphalytics, a comprehensive benchmark for graph analysis frameworks that facilitates the comparison of performance between these frameworks. Second, we design, implement, and evaluate Grade10, a system for automated, in-depth performance analysis of graph analysis frameworks. Through experimental evaluation of the Graphalytics ecosystem, we gain insight into the performance of six modern graph analysis frameworks.

The proliferation of big-data processing platforms has already led to radically different system designs, such as MapReduce and the newer Spark. Understanding the workloads of such systems enables tuning and could foster new designs. However, whereas MapReduce workloads have been characterized extensively, relatively little public knowledge exists about the characteristics of Spark workloads in representative environments. In this work, we focus on understanding the behavior and cache performance of the storage sub-system used for Spark workloads in the cloud. First, we statistically characterize its usage. Second, we design a generative model to tackle the scarcity of workload traces. Third, we design a cache policy putting our insight from the characterization to work. Finally, we evaluate the performance of different cache policies for big data workloads via simulation.

Minecraft is one of the most popular games of all time,
and provides both entertainment and education to people around the globe.
One of the main challenges in gaming is increasing the scalability of these games to support the large numbers of players they attract.
Minecraft-like games make this challenge even more difficult by introducing the novel feature of a modifiable world.
The world that players explore can be fully modified, allowing players to build anything from a recreation of Manhattan to a functioning digital computer.
These modifications increase the need for communication between players, limiting scalability.
We design and evaluate Meerkat, a prototype system that uses Dynamic Conits to reduce this communication and increase the scalability of Minecraft-like games.

Graphs are becoming more popular day by day. This has lead to the development of different graph-processing platforms, such as Hadoop, Apache Giraph and GraphLab. With this wide variety of graph processing platforms, which one should we choose for a specific use case? Which platform has the best performance? To solve this issue, we can benchmark the different platforms. Our client developed a benchmark suite for graph processing platforms: the LDBC Graphalytics. Users can run the benchmark on the platforms and benchmark result files are generated. This result file contains thousands of lines of information. This makes comparing the benchmark result files not practical. A way to compare the benchmark results easily is to make the platforms compete in competitions. The platforms are then ranked based on their performance. For this, a website is needed to hosts the competitions. In this project, we have developed a global competition website that serves as a solution.

Complex workflows that process sensor data are useful for industrial infrastructure management and diagnosis. Although running such workflows in clouds promises reduces operational costs, there are still numerous scheduling challenges to overcome. Such complex workflows are dynamic, exhibit periodic patterns, and combine diverse task groupings and requirements. In this work, we propose ANANKE, a scheduling system addressing these challenges. Our approach extends the state-of-the-art in portfolio scheduling for data centers with a reinforcement-learning technique and proposes various scheduling policies for managing complex workflows. Portfolio scheduling addresses the dynamic aspect of the workload. Reinforcement learning, based in this work on Q-learning, allows our approach to adapt to the periodic patterns of the workload, and to tune the other configuration parameters. The proposed policies are heuristics that guide the provisioning process, and map workflow tasks to the provisioned cloud resources. Through real-world experiments based on real and synthetic industrial workloads, we analyze and compare our prototype implementation of ANANKE with a system without portfolio scheduling (baseline) and with a system equipped with a standard portfolio scheduler. Overall, our experimental results give evidence that a learning-based portfolio scheduler can perform better (5--20%) and cost less (20--35%) than the considered alternatives.

Business processes are part of every company’s daily business. In the past hundreds of years, almost every aspect in the operation of businesses has seen a two-pronged transformation: first, toward defining and then following processes, and second, toward making the processes efficient through the use of technology.
In this work, we define a technology-driven approach to business processes: Business Process as a Service. Starting from the current technological possibilities and architectural trends, we define BPaaS, and list requirements for a functional BPaaS system in enterprise settings. Key to our requirements, we consider (1) business process creation, execution, and monitoring, (2) the pay-per-use business model, and (3) useful trade-offs in the cost-scalability space. Using these requirements, we design a reference architecture for BPaaS.

We implement the reference architecture as a prototype BPaaS solution, Elastic-BPM. The key feature of Elastic-BPM is efficient operation, which it achieves by enabling the elastic use of infrastructure. Key components of our reference architecture are (1) the user dashboard and process manager, responsible for the creation, execution, and monitoring of business processes, (2) the virtual infrastructure making use of small, preemptable components, to enable the pay-per-use business model, and (3) the scheduler, responsible for the trade-offs in the cost-scalability space.

The implementation is based on and makes use of state-of-the-art cloud components, such as Angular for the dashboard, Elasticsearch for the scheduler input, and Docker for the virtual infrastructure. We further equip the scheduling component of Elastic-BPM with three different dynamic provisioning algorithms; Static, OnDemand, and Learning, which cover conceptually one static and two dynamic approaches to provisioning infrastructure, respectively.

We validate Elastic-BPM by deploying it both locally and in the Microsoft Azure cloud, and by running over 50 experiments using a synthetic but diverse workload. We find not only optimal setting for the cloud components in the system, but also for scheduling the human resources partaking in the business process.