Distributed Dataflow Transactions

Abstract

As serverless computing grows in popularity, developers are demanding more from existing serverless models. One example is the emergence of Stateful Function as a Service (SFaaS), in which state is added to operators in existing Function as a Service (FaaS) models, to support microservice-type applications while utilising the benefits of a serverless architecture. However, current SFaaS systems cannot provide performant transactions across operators with strong semantics. In this thesis, we present three conceptual transaction protocols for SFaaS dataflow systems based on Two-phase Commit (2PC), Deterministic Databases and Conflict-free Replicated Datatype (CRDT)s. Based on these insights, we implement Rhea, a deterministic transaction protocol in the prototype SFaaS execution engine, Universalis. Two optimizations are implemented for Rhea: deterministic reordering and a fallback mechanism, to reduce aborts caused by Read-after-write (RAW) and Write-after-write (WAW) dependencies, respectively. We present a transaction benchmarking client for Universalis that supports workloads from the Transaction Processing Performance Council Benchmark C (TPC-C) and the Yahoo! Cloud Serving Benchmark (YCSB). Using the client, we compare Rhea to a 2PC baseline microservice application. We found that Rhea can provide more than twice the throughput and half the latency of 2PC in the baseline application across a variety of workloads.