Distributed Dataflow Transactions
A.C. Walker (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Asterios Katsifodimos – Mentor (TU Delft - Web Information Systems)
Kyriakos Psarakis – Coach (TU Delft - Web Information Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
As serverless computing grows in popularity, developers are demanding more from existing serverless models. One example is the emergence of Stateful Function as a Service (SFaaS), in which state is added to operators in existing Function as a Service (FaaS) models, to support microservice-type applications while utilising the benefits of a serverless architecture. However, current SFaaS systems cannot provide performant transactions across operators with strong semantics. In this thesis, we present three conceptual transaction protocols for SFaaS dataflow systems based on Two-phase Commit (2PC), Deterministic Databases and Conflict-free Replicated Datatype (CRDT)s. Based on these insights, we implement Rhea, a deterministic transaction protocol in the prototype SFaaS execution engine, Universalis. Two optimizations are implemented for Rhea: deterministic reordering and a fallback mechanism, to reduce aborts caused by Read-after-write (RAW) and Write-after-write (WAW) dependencies, respectively. We present a transaction benchmarking client for Universalis that supports workloads from the Transaction Processing Performance Council Benchmark C (TPC-C) and the Yahoo! Cloud Serving Benchmark (YCSB). Using the client, we compare Rhea to a 2PC baseline microservice application. We found that Rhea can provide more than twice the throughput and half the latency of 2PC in the baseline application across a variety of workloads.