Tydi is an open specification for streaming dataflow designs in digital circuits, allowing designers to express how composite and variable-length data structures are transferred over streams using clear, data-centric types. This provides a higher-level method for defining interfaces between components as opposed to existing bit- and byte-based interface specifications.

In this thesis, an open-source intermediate representation (IR) is introduced which allows for the declaration of Tydi's types. The IR enables creating and connecting components with Tydi Streams as interfaces, called Streamlets. It also lets backends for synthesis and simulation retain high-level information, such as documentation. Types and Streamlets can be easily reused between multiple projects, and Tydi’s streams and type hierarchy can be used to define interface contracts, which aid collaboration when designing a larger system.

The IR codifies the rules and properties established in the Tydi specification and serves to complement computation-oriented hardware design tools with a data-centric view on interfaces. To support different backends and targets, the IR is focused on expressing interfaces, and complements behavior described by hardware description languages and other IRs. Additionally, a testing syntax for the verification of inputs and outputs against abstract streams of data, and for substituting interdependent components, is presented which allows for the specification of behavior.

To demonstrate this IR, a grammar, parser, and query system have been created, and paired with a backend targeting VHDL.

vailability of FPGAs is increasing due to cloud service offerings. In the wake of a new in-memory storage format specification, Apache Arrow, FPGAs are increasingly interesting for data processing acceleration in the big data domain. The Fletcher framework can be used to easily develop FPGA accelerated applications that access data stored in Apache Arrow format, while providing throughput near the system’s limit. The current implementation of Fletcher has limited support for memory management of FPGA-local memory, with one of the biggest limitations being that memory can only be used once.
This thesis explores several memory management techniques which could be suitable for use on FPGAs in a big data context. Paged memory is implemented on FPGA within the Fletcher framework in order to facilitate this memory management. The implemented system takes less than 5 % of a data centre FPGA card’s resources (Xilinx UltraScale+ VU9P). Experiments show that the paged memory provides throughput of over 99.7 % of the system’s throughput for linear memory accesses. Random memory access throughput for paged memory drops to between 30 % and 90 % of the system’s original throughput, depending on request size. The performance drop can be lightened or even prevented by employing suitable address-translation caches.

With the advent of high-bandwidth non-volatile storage devices, the classical assumption that database analytics applications are bottlenecked by CPUs having to wait for slow I/O devices is being flipped around. Instead, CPUs are no longer able to decompress and deserialize the data stored in storage-focused file formats fast enough to keep up with the speed at which compressed data is read from storage. In order to better utilize the increasing I/O bandwidth, this work proposes a hardware accelerated approach to converting storage-focused file formats to in-memory data structures. To that end, an FPGA-based Apache Parquet reading engine is developed that utilizes existing FPGA and memory interfacing hardware to write data to memory in Apache Arrow's in-memory format. A modular and expandable hardware architecture called the ParquetReader with out-of-the-box support for DELTA_BINARY_PACKED and DELTA_LENGTH_BYTE_ARRAY encodings is proposed and implemented on an Amazon EC2 F1 instance with an XCVU9P FPGA.

The ParquetReader has great area efficiency, with a single ParquetReader only requiring between 1.18% and 2.79% of LUTs, between 1.27% and 2.92% of registers and between 2.13% and 4.47% of BRAM depending on the targeted input data type and encoding. This area efficiency allows for instantiating a large number of (possibly different) ParquetReaders for parallel workloads. Multiple Parquet files of varying types and encodings were generated in order to measure the performance of the ParquetReaders. A single engine has achieved up to 2.81x speedup for DELTA_LENGTH_BYTE_ARRAY encoded strings and 2.79x speedup for DELTA_BINARY_PACKED integers when compared to CPU-only Parquet reading implementations, attaining a throughput between 2.3 GB/s and 7.2 GB/s (limited by the interface bandwidth of the testing system) depending on the input data. The high throughput and low resource utilization of the ParquetReader allow for the interface bandwidth to be saturated using multiple ParquetReaders utilizing only a small amount of the FPGA's resources.