Experimental evaluation of distributed similarity joins in stream processing environments

Similarity joins are operations which involve identifying similar pairs of records within one or multiple datasets. These operations are typically time-sensitive, as timely identification of relations can lead to increased profitability. Therefore, it is advantageous to analyze them using a stream processing system, which offers real-time...

Towards Evaluating Stream Processing Autoscalers

In this work, we evaluate autoscaling solutions for stream processing engines. Although autoscaling has become a mainstream subject of research in the last decade, the database research community has yet to evaluate different autoscaling techniques under a proper benchmarking setting and evaluation framework. As a result, every newly proposed...

SECLEDS: Sequence Clustering in Evolving Data Streams via Multiple Medoids and Medoid Voting

Sequence clustering in a streaming environment is challenging because it is computationally expensive, and the sequences may evolve over time. K-medoids or Partitioning Around Medoids (PAM) is commonly used to cluster sequences since it supports alignment-based distances, and the k-centers being actual data items helps with cluster...

Anyone Can Cloud: Democratizing Cloud Application Programming

The cloud is widely adopted as a flexible and on-demand computing infrastructure. In recent years, a new and promising cloud paradigm emerged: serverless computing. Serverless computing promises a pay-as-you-go model and offers features such as autoscaling and high availability. Nevertheless, developing scalable cloud applications remains a...

State Migration in Stream Processing Systems

In recent years, the interest for serverless computing has grown tremendously. The most common form of serverless computing, Function-as-a-Service (FaaS), uses data centers of large public cloud providers to run simple functions. The cloud providers are responsible for the operational and deployment aspects. Non-trivial function implementations...

S-Query: Opening the Black Box of Internal Stream Processor State

At the moment we are witnessing the maturation of distributed streaming dataflow systems whose use-cases have departed from the mere analysis of streaming windows and complex-event processing, as they now extend to cloud applications, workflows and even e-commerce. The state of streaming operators has been so far hidden from external...

Clonos: Consistent Causal Recovery for Highly-Available Streaming Dataflows

Stream processing lies in the backbone of modern businesses, being employed for mission critical applications such as real-time fraud detection, car-trip fare calculations, traffic management, and stock trading. Large-scale applications are executed by scale-out stream processing systems on thousands of long-lived operators, which are subject...

The Design and Implementation of a Stateful Streaming Rule Language for the Maritime Sector

Stream processing has taken a prominent position in the software engineering industry. Many applications, from a small to large scale, embrace this paradigm to deal with the difficulties of responding to events that happen asynchronously: programmers define "operators" that perform small, independent tasks on individual events. These operators...

A plug-in infrastructure for the CodeFeedr project

CodeFeedr is a research project at the software engineering division of the Delft University of Technology in collaboration with the Software Improvement Group. The research focuses on a software infrastructure which serves software practitioners in utilizing data-driven decision making. Currently, frameworks like Apache Flink are capable of...

Data streams in spreadsheets with reactive extensions

The amount of collected data increases exponentially and this exponential growth comes with a demand to analyze big volumes of data and to process the data fast and in real time. Business analysts have the expertise in the domain of the desired stream processing applications, but often lack the technical skills to create these kind of...