FPGAs in Big Data: On the transparent and efficient acceleration of big data frameworks

The increasing volume and latency requirements of big data impose challenges on the processing capacity of existing computing systems. FPGA accelerators can be leveraged to overcome these challenges, but questions remain as to how these accelerators are best deployed to accelerate big data frameworks. This work investigates how future big data...

Battling the CPU Bottleneck in Apache Parquet to Arrow Conversion Using FPGA

In the domain of big data analytics, the bottleneck of converting storage-focused file formats to in-memory data structures has shifted from the bandwidth of storage to the performance of decoding and decompression software. Two widely used formats for big data storage and in-memory data are Apache Parquet and Apache Arrow, respectively. In...

Tens of gigabytes per second JSON-to-Arrow conversion with FPGA accelerators

JSON is a popular data interchange format for many web, cloud, and IoT systems due to its simplicity, human readability, and widespread support. However, applications must first parse and convert the data to a native in-memory format before being able to perform useful computations. Many big data applications with high performance requirements...

Generating high-performance FPGA accelerator designs for big data analytics with Fletcher and Apache Arrow

As big data analytics systems are squeezing out the last bits of performance of CPUs and GPUs, the next near-term and widely available alternative industry is considering for higher performance in the data center and cloud is the FPGA accelerator. We discuss several challenges a developer has to face when designing and integrating FPGA...

Dataflow Hardware Design for Big Data Acceleration Using Typed Interfaces

Recent trends in large-scale computing demonstrate continuous growth in the need for raw processing performance. At the same time, the slowdown of vertical scaling pushes the industry towards more energy-efficient heterogeneous architectures. With the appearance of FPGAs in the cloud and data centers, a new architecture is offered for offloading...

Transparently Accelerating Spark SQL Code on Computing Hardware

Through new digital business models, the importance of big data analytics continuously grows. Initially, data analytics clusters were mainly bounded by the throughput of network links and the performance of I/O operations. With current hardware development, this has changed, and often the performance of CPUs and memory access became the new...

High-Throughput Big Data Analytics Through Accelerated Parquet to Arrow Conversion

With the advent of high-bandwidth non-volatile storage devices, the classical assumption that database analytics applications are bottlenecked by CPUs having to wait for slow I/O devices is being flipped around. Instead, CPUs are no longer able to decompress and deserialize the data stored in storage-focused file formats fast enough to keep up...

Supporting Columnar In-memory Formats on FPGA: The Hardware Design of Fletcher for Apache Arrow

As a columnar in-memory format, Apache Arrow has seen increased interest from the data analytics community. Fletcher is a framework that generates hardware interfaces based on this format, to be used in FPGA accelerators. This allows efficient integration of FPGA accelerators with various high-level software languages, while providing an easy-to...