Benchmarking Stateful Serverless Functions

Abstract

Serverless computing is an increasingly popular paradigm in cloud computing where many of the operational challenges of running cloud applications, like server provi- sioning and management, are left to the cloud provider. A popular form of server- less computing is Functions-as-a-Service (FaaS), where the user submits functions for which the resources are automatically provisioned and scaled. However, FaaS functions are traditionally stateless, and thus rely on external services to handle state. Stateful FaaS systems are an extension to traditional FaaS offerings which have built-in function state management and function addressability, which allows for functions to communicate and to form complex operations. This work introduces a performance benchmark for stateful functions systems, based on an e-commerce application. The benchmark includes two workflows based on two complex oper- ations that involve multiple stateful functions. The workflows can be dynamically altered to form different function calling structures. We provide reference imple- mentations of the benchmark application for four current stateful FaaS systems, and a benchmark client which runs the two benchmark workloads on these implementa- tions. We show that our benchmark can be used to test and compare stateful systems on performance, networking, cost and scalability, by running various experiments on our reference implementations.