The Design and Implementation of a Stateful Streaming Rule Language for the Maritime Sector

Abstract

Stream processing has taken a prominent position in the software engineering industry. Many applications, from a small to large scale, embrace this paradigm to deal with the difficulties of responding to events that happen asynchronously: programmers define "operators" that perform small, independent tasks on individual events. These operators are then connected to form a (usually asyclic) network of tasks performed on one or multiple streams. While very useful as can be witnessed by the vast amount of stream processing tools available to the industry, one problem remains difficult to tackle in this model: state. When comparing support for stateful operations in matured stream-processing systems, one will find that none of the systems agree on how state should be modeled. Some systems outright ignore state, while others provide ever different APIs to manage it. In this thesis, we present a small but versatile model that combines traditional stream processing with stateful operations from relational algebra, providing one uni- fied way of thinking about streams and state. To substantiate the models practicality, we present an end-to-end implementation of a real-world case study for Europe’s largest shipping container port, along with a performance analysis. Moreover, we assert that this model is not just tied to one specific stream-processing solution, but can be implemented with ease on any actor-based platform.