Apache Flink™

Stream and Batch Processing in a Single Engine

Journal article (2015)

Authors

Paris Carbone Swedish Institute of Computer Science, KTH Royal Institute of Technology
A Katsifodimos German Research Centre for Artificial Intelligence (DFKI), Technical University of Berlin
Stephan Ewen Data Artisans GmbH
Volker Markl Technical University of Berlin, German Research Centre for Artificial Intelligence (DFKI)
Seif Haridi Swedish Institute of Computer Science, KTH Royal Institute of Technology
Kostas Tzoumas Data Artisans GmbH
Affiliation
External organisation
More Info
expand_more

Published Date

2015

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Affiliation

External organisation

Abstract

Apache Flink is an open-source system for processing streaming and batch data. Flink is built on the philosophy that many classes of data processing applications, including real-time analytics, continuous data pipelines, historic data processing (batch), and iterative algorithms (machine learning, graph analysis) can be expressed and executed as pipelined fault-tolerant dataflows. In this paper, we present Flink’s architecture and expand on how a (seemingly diverse) set of use cases can be unified under asingle execution model.