State Migration in Stream Processing Systems

Master thesis (2021)

Authors

M.J. Ploemen Electrical Engineering, Mathematics and Computer Science

Contributors

A Katsifodimos Web Information Systems - EEMCS (mentor)
M. Fragkoulis Web Information Systems - EEMCS (mentor)
A. van Deursen Software Technology (graduation committee member)
Faculty
Electrical Engineering, Mathematics and Computer Science, Electrical Engineering, Mathematics and Computer Science
Copyright: © 2021 Marlo Ploemen
More Info
expand_more

Published Date

19-10-2021

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

In recent years, the interest for serverless computing has grown tremendously. The most common form of serverless computing, Function-as-a-Service (FaaS), uses data centers of large public cloud providers to run simple functions. The cloud providers are responsible for the operational and deployment aspects. Non-trivial function implementations require state to perform the desired business logic. Current FaaS implementations using an externalized database for state cannot achieve the low latency scenarios required for some services. Previous work investigated Stateful Function-as-a-Service (SFaaS) using Stream Processing Systems as a runtime. State migration, as a result of schema evolution on SFaaS, remains an open challenge.

This thesis investigates common practices regarding schema evolution and their applicability to stream processing systems. Based on the investigation, the performed work demonstrates a schema driven approach to state migration in stream processing systems. The approach demonstrates that a view on both the source and target state schema can also yield implicit transformations for schema compatibility.

The work is demonstrated using a modified version of Apache Flink and evaluated based on common evolution scenarios and hypothesized changes to real world queries from the NEXmark benchmark.