Supporting Columnar In-memory Formats on FPGA

The Hardware Design of Fletcher for Apache Arrow

Conference paper (2019)

Authors

J.W. Peltenburg Computer Engineering - EEMCS
J. van Straten Computer Engineering - EEMCS
M. Brobbel Computer Engineering - EEMCS
Z. Al-Ars Computer Engineering - EEMCS
Research Group
Computer Engineering (EEMCS) (TU Delft)
Copyright: © 2019 J.W. Peltenburg, J. van Straten, M. Brobbel, H.P. Hofstee, Z. Al-Ars
DOI
help_outline
https://doi.org/10.1007/978-3-030-17227-5_3
FPGA Apache Arrow Fletcher
More Info
expand_more

Published Date

2019

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Quantum & Computer Engineering

Research Group

Computer Engineering

Abstract

As a columnar in-memory format, Apache Arrow has seen increased interest from the data analytics community. Fletcher is a framework that generates hardware interfaces based on this format, to be used in FPGA accelerators. This allows efficient integration of FPGA accelerators with various high-level software languages, while providing an easy-to-use hardware interface for the FPGA developer. The abstract descriptions of data sets stored in the Arrow format, that form the input of the interface generation step, can be complex. To generate efficient interfaces from it is challenging. In this paper, we introduce the hardware components of Fletcher that help solve this challenge. These components allow FPGA developers to express access to complex Arrow data records through row indices of tabular data sets, rather than through byte addresses. The data records are delivered as streams of the same abstract types as found in the data set, rather than as memory bus words. The generated interfaces allow for full system bandwidth to be utilized and have a low area profile. All components are open sourced and available for other researchers and developers to use in their projects.