Qbeast is an open-source extension to Apache Spark and Delta Lake that maintains a multidimensional OTree index over columnar tables, enabling approximate query processing through file pruning. Each time a table is queried, the system constructs an \emph{IndexStatus}, a compact in-memory map from cube metadata. It does so through two sequential steps: replaying the Delta Lake transaction log to enumerate the active file set ($O(D)$ in log depth $D$), then running a distributed \texttt{groupBy} aggregation over all active block metadata ($O(N)$ in active file count $N$). The log replay cost is paid once per relation, the aggregation is rerun on every query against the same relation. Delta Lake's periodic checkpoint compaction caps the log-replay cost at a bounded overhead; the per-query $0(N)$ aggregation is the gap that smart checkpointing fills.

This thesis introduces \emph{smart checkpointing}: a persistent cache for the IndexStatus, saved as a $K$-row Parquet file (where $K$ is the number of OTree cubes, $K \ll N$) and written automatically after each batch of writes. Further queries load the snapshot and replay the $G$ commits that have accumulated since the snapshot. This reduces the per-query reconstruction cost from $O(N)$ to $O(K + \alpha G)$. An incremental merging algorithm handles the common pure-append deltas and is equivalent to the full index rebuild. A closed-form cost model is defined, which gives an optimal snapshot interval $G^* = \sqrt{F_\text{write} \cdot C_\text{dump} / (F_\text{query} \cdot \alpha)}$, providing a workload-based tuning rule.

Experiment~1 establishes the baseline $O(N)$ coefficient $E_\text{shuffle} = 0.353$\,ms/file ($R^2 = 0.950$) over a wide range of active file counts. Experiment~2 validates the smart path with $\alpha \approx 21$\,ms/commit ($R^2 > 0.99$), confirms that $\alpha$ is independent of table size across a range of $N$, and derives $G^* \approx 10$\,commits. Experiment~3 validates the amortized cost model under mixed read/write workloads. For read-heavy workloads, total operational cost is reduced by 45\% at $G=500$. Experiment 4 emulates a deployment on a distributed cluster by adding a per-file latency. With a representative latency of $E_\text{net} = 20$ the emulated speedup is up to $19\times$ , amplifying the advantage of smart checkpointing. The final experiment replicates two experiments on real-world datasets, to confirm values from synthetic experiments.

The central finding of the thesis is the transformation of the index reconstruction cost from a full data history rebuild to a snapshot and delta load. The practical benefit starts with approximately 30 prior commits in cold-start scenarios, and grows with a big table in a distributed deployment.
... Read more

Stateful Functions-as-a-Service (SFaaS) platforms, such as Styx, are emerging as powerful abstractions for building distributed, serverless cloud applications. By combining the abilities of FaaS with strong transactional guarantees, they enable complex, stateful workflows without requiring developers to manage infrastructure. However, they lack built-in support for analytical queries across distributed function state. This thesis addresses that gap by proposing H-Styx, whose hybrid architecture extends Styx with a snapshot-based Query Engine, enabling near-real-time OLAP queries over global state while maintaining performance isolation for transactions. The Query Engine integrates seamlessly into the Styx architecture, leveraging periodic snapshots transmitted via a loosely-coupled, asynchronous interface. It ingests partitioned state from object store MinIO into columnar database DuckDB, supports incremental delta loads, and delivers results over a Kafka-based interface to achieve scalable, low-latency analytical querying while employing robust fault tolerance.

Empirical evaluation demonstrates that H-Styx preserves transactional throughput and latency under hybrid workloads, while significantly outperforming a baseline HTAP architecture (Postgres with Streaming Replication) on analytical throughput and providing superior workload isolation. These results validate the feasibility of supporting hybrid transactional and analytical processing in SFaaS environments. Overall, H-Styx bridges a crucial capability gap in SFaaS, enabling more powerful data-driven applications in distributed, event-driven architectures. ... Read more

Building scalable and consistent cloud applications is notoriously difficult due to the challenges of state management and execution consistency in distributed environments. Functions-as-a-Service (FaaS) platforms offer flexible scalability, but weak execution guarantees forces engineers to mix business logic with infrastructure concerns, adding error-handling code, retry mechanisms and consistency checks throughout their applications. At the same time, dataflow systems like Apache Flink offer exactly-once semantics, but their functional APIs often conflict with the imperative, object-oriented style preferred by mainstream developers.

This work aims to address this disconnect, arguing that modern transactional applications, from e-commerce to payment systems to business workflows, naturally form stateful dataflow graphs. By allowing developers to write familiar imperative code that executes on dataflow systems with strong consistency guarantees, we could eliminate the need to handle many infrastructure concerns explicitly.

To this end, we introduce Cascade, a compiler pipeline and intermediate representation that bridges the gap by translating imperative Python code into stateful, parallelizable dataflow graphs. Cascade extends prior work by providing a representation that is both expressive and optimizable, and we demonstrate optimizations including parallel execution via data dependency analysis and dynamic value prefetching. Our results show significant performance gains with these optimizations, all while maintaining the strong execution guarantees of the underlying execution target. Finally, we offer avenues for future research by discussing further optimization possibilities and extensions within our proposed framework. ... Read more

Distributed systems are vital for handling large-scale data and rely on geo-distributed databases to ensure low latency and high availability. Traditional benchmarks, such as TPC-C and YCSB-T, are not designed to handle the complexities of geo-distributed environments and do not allow for configuration of multi-home transaction ratios or dynamic data access patterns. To fill this gap, we implement a benchmark based on the MovR workload and assess its performance on the Detock, Janus, SLOG, and Calvin geo-distributed database systems. Key insights revealed through experiments are that network conditions act as a major bottleneck and high concurrency leads to unsustainable latency spikes which severely limits scalability. ... Read more