Existing evaluations of geo-distributed databases still rely almost exclusively on standard limited workloads such as TPC-C and YCSB+T, which reveal little information about the true cost of wide-area coordination. In this paper, we present a configurable benchmarking framework b
...
Existing evaluations of geo-distributed databases still rely almost exclusively on standard limited workloads such as TPC-C and YCSB+T, which reveal little information about the true cost of wide-area coordination. In this paper, we present a configurable benchmarking framework built around the Product-Parts-Supplier (PPS) workload, and use it to evaluate four representative systems that support geo-distributed transactions: Calvin, SLOG, Detock, and Janus. The experiments run across six realistic and insightful scenarios that vary the transactional load, contention, client count, regional bias, network latency, and packet loss.
The results uncover clear design trade-offs between the systems and demonstrate that our new framework is capable of filling several evaluation holes left by the standard workloads. Our framework introduces important features such as dependent transactions that may abort and retry, longer and tunable read/write sets, and fine-grained control over which regions will participate in the commit. However, the PPS workload comes with some limitations, and thus, the framework does not cover every case. As a consequence, for now, it should complement TPC-C and YCSB+T, not replace them.