ZA

Z. Al-Ars

info

Please Note

37 records found

Aiding financial crime detection in large datasets

Master thesis (2026) - K. Khalili, H.P. Hofstee, Kubilay Atasu, Z. Al-Ars
With the rise of memory costs and the persistent under-utilization of memory in clusters, researchers have begun exploring alternative approaches to improve memory efficiency and reduce operational costs. Resource disaggregation is becoming increasingly common and sought after, driven by the emergence of new interconnect standards such as CXL and, previously, OpenCAPI. While the industry is primarily moving toward memory pooling, where memory is dynamically provisioned among applications or virtual machines, this work investigates distributed memory disaggregation and sharing. IBM's Power10 processors include hardware support that enables multiple systems to directly share memory. However, few applications have been developed to take advantage of disaggregated shared memory.

Since Memory Inception, Power10 processors' memory disaggregation hardware, is not yet fully operational, a ThymesisFlow prototype, upgraded to support a shared disaggregated memory system with the help of Apache Arrow, is used to implement a practical application. The selected application is a graph processor capable of detecting money laundering patterns in financial transaction graphs in real-time. These patterns yield transaction features that machine learning algorithms can use to identify fraudulent financial transactions.

Our proof-of-concept implementation enables the creation of a distributed graph, represented as Apache Arrow tables, that can process large datasets in real-time. The graph resides in a shared disaggregated memory region and can be accessed by multiple systems without data copying, incurring lower latency penalties than network-based data retrieval. The distributed graph processor was developed and tested using the ThymesisFlow prototype provided by the Hasso Plattner Institute. ...
Master thesis (2024) - R. Feng, Q. Wang, Z. Al-Ars
There are increasing applications of Visible Light Communication, and LED-to-Camera communication is one of the promising research directions. In this paper, we explore LED-to-Camera communication in mobile scenarios based on the rolling shutter effect to get higher data transmission rate. Movement causes the deformation of the overall shape of the LED in the captured image. And stripes close to the LED edges are too dark to be detected. Both affect the data detection by utilizing the rolling shutter effect. A method is proposed that can increase the transmission rate, which solves the problems caused by movement and darkness of stripes that close to the LED cover edges. After processing, the region of interest is recouped, i.e., the processing range used to detect the rolling shutter stripes becomes larger, which contains more complete data. An approach to detect stripe edges by converting pixel information into binary signal is proposed, which is used to solve the problem of undetectable stripe edges due to high frequency. The proposed method is evaluated through experiments under different conditions. The results demonstrate that the method is effective for addressing the challenges of LED-to-camera communication in mobile scenarios. ...

Providing a framework for data center parameter analysis

Master thesis (2024) - F. Kerkhof, E. Smeitink, Z. Al-Ars, E.F.M. van Boven, J.S. Cox, D.J. Tuinhof
The demand for computational resources is increasing exponentially due to an increasing amount of digital services. Cloud computing is becoming the standard for enterprises to provide these resources. This resulted in hyperscalers which consist of a large number of servers. Data centers consume more than 1% of the world’s electrical energy. Therefore, many techniques are developed that both help to fulfil the needs of digital services and reduce the energy consumption of data center services. Modern-day servers can switch between different operating states which are often integrated in a power configuration mode of servers known as eco-mode. One of these techniques throttles the clock frequency of a central processing unit (CPU) which enables the possibility to lower the power needed for that CPU. This technique is known as dynamic frequency and voltage scaling (DFVS) and the states it switches between are known as performance states (P-states). A different technique that is integrated with eco-mode is the ability to switch between different idle states of the CPU. These states define whether certain caches of the CPU are flushed or not to conserve energy. These states are known as core states (C-states). A different approach that is focused on conserving energy is virtualisation within data centers. Virtualisation enables one physical server to host multiple virtual instances of servers (virtual machines). This reduces resource wastage and energy consumption of a data center. However, this creates the need for a strategic placement that ensures that the demands of the virtual machines are met and that minimises energy consumption and resource wastage. This thesis analyses four of these techniques: the best fit decreasing (BFD) algorithm, the integer linear programming (ILP) algorithm, the particle swarm optimisation (PSO) algorithm and the genetic algorithm (GA). This thesis provides a framework that uses a holistic approach to provide insights into the effects of using eco-mode of servers within the dynamics of virtual machine placement in data centers. This framework serves as a first step in parameterising the dynamics of a data center regarding its energy consumption and performance. The results show a potential energy reduction of up to approximately 20% with negligible impact on a data center’s performance. This result occurs when applying the best fit decreasing algorithm and having a server with an energy-efficient eco-mode. However, this thesis does not cover all parameters that play a role in the data center’s performance and energy consumption, so more research on this area is recommended. ...
As digital communication networks become integral to society and the economy—driving everything from personal communication to business operations and smart city infrastructure—the capacity of these networks must continuously expand to accommodate higher data volumes. The Ericsson Mobility Report demonstrates the growing need for higher throughput and increased network capacity to meet end-user expectations. One of the viable approaches to increase network capacity is through the deployment of small cells.
In general, the related axioma is: the higher the offered bandwidth (higher radio frequencies), the shorter the radio ranges will be. Small cells provide additional capacity (preferably) by means of high-frequency bands such as 3.5 GHz and 26 GHz offloading the macro network in particular short-range hotspots. Small cells can deliver higher data rates to end users, making their deployment essential at hotspots in densely populated urban areas where additional capacity will be required. This master’s thesis evaluates Radio Access Network (RAN) architecture options for combining macro cells and small cells from both theoretical and practical perspectives.

The main finding from the Radio Access Network-related theoretical analysis is that the RAN architecture split option 7.2x recommended by the O-RAN Alliance is the optimal solution for indoor and outdoor deployment of small cells. Split option 7.2x offers significant benefits, such as minimising the impact on transport bandwidth while enhancing the virtualisation capabilities of the gNB Central Unit (CU) and Distributed Unit (DU), and enabling a cost-effective design of the Radio Unit (RU).


To gain an understanding of the practicalities involved in small cell deployment, this master thesis, through the Utrecht practical case study, examines potential locations for the installation of RUs, DUs, and CUs using a combination of expert interviews, Google Street View analysis, QGIS visualisations, and site visits. Proposed locations for RUs include three tall lamp posts and two security camera poles, while wharf cellars managed by Stedin and a macro cell base station are recommended for the installation of CUs and DUs. These recommendations are based on an analysis of power availability, transport, site accessibility and expert interviews. From the research it is concluded that small cell deployment in the Utrecht researched area is feasible, provided that specific challenging boundary conditions are met such as collaboration among MNOs and the use of existing poles.

The practical part of this thesis research also delves into the complex interplay of eight socioeconomic sectors and their roles in small cell deployment, providing insights into the trans-sector nature of this project.
The main results from this practical part of the research concern the conclusions from the Utrecht practical case study about realizing small cells:
The main results from this practical part of the research concern the conclusions from the Utrecht practical case study about realizing small cells:
1. The rollout of small cells is a complex multi-actor value case with substantially more actors to collaborate in comparison with rolling out macro cells by primarily telecom operators and municipalities (issuing licenses).
2. Municipalities are best suited to fulfil the role of orchestrator in the rollout of small cells because they can coordinate diverse stakeholders to ensure seamless integration with existing infrastructure, maintain city aesthetics and establish a direct line of communication with residents for feedback and service improvement.
Based on some representative calculations of small cell deployments in the city of Utrecht we obtained good insight into the complexity as well as cost of small cells. A few of the quantitative results are mentioned below:
•The price of a small cell is roughly around €60,000 per small cell site, including digging for transmission and power costs.
•In case the fronthaul is replaced by a wireless connection, roughly 30% of cost savings can be achieved.

...
Master thesis (2023) - D.J.A. de Graaf, A. van Deursen, S.S. Chakraborty, N. Yakovets, Z. Al-Ars
Graph databases are systems to efficiently store and query large graphs. As graph databases grow in popularity, they are used to answer increasingly diverse and complex queries. However, graph databases typically have a very limited query language that cannot express arbitrary algorithms. As a result, many users treat the database as a storage layer to export data from and develop algorithms in external tools, wasting computation power and storage space.

We present graphalg, a high-level, domain-specific language for writing graph algorithms embedded into traditional graph queries. Our language is based on linear algebra, with a syntax resembling GraphBLAS, and implemented in the AvantGraph database.

We implement a compiler for graphalg that can target an interpreter built on top of a GraphBLAS implementation. Alternatively, our compiler can transform graphalg programs into a relational algebra with loops, unifying the representation of query and algorithm. We evaluate the programmability and performance of our system on the GAP Benchmark Suite for graph algorithms. Our language is expressive enough to concisely represent all GAP benchmark programs, with the majority of programs achieving performance comparable to an optimized C implementation.

We conclude that graph algorithm support can be integrated into graph databases to increase their programmability. Running graph algorithms inside of the database increases performance and reduces memory consumption compared to using external tools for the analysis. Rather than thinking of graph databases as limited tools for answering simple queries, we demonstrate that they can instead be a programmable framework for efficient large-scale data analysis. ...
Master thesis (2023) - Z. LIU, S.S. Chakraborty, A. van Deursen, Z. Al-Ars
Atomics is an important primitive for programming languages like C++ to develop concurrent software. Atomic variables, together with weak memory models allow for a bigger space for instruction reordering and compiler optimizations. However, the current compilers like LLVM do not support many transformations of atomics, which may lose chances of optimizations.

In this thesis project, we built a compiler optimizing C++ atomic memory accesses based on LLVM 14.0.0. We modified related LLVM passes to enable these optimizations. Specifically, our compiler is able to optimize Read-After-Read(RAR), Read-After-Write(RAW), and Overwritten Write(OW) patterns containing atomics. To achieve this, we removed checks in LLVM that forbid atomic accesses from being processed. And we added constraints and adapted them into existing algorithms of LLVM passes, to ensure the soundness of our transformations.

We tested our compiler using randomly generated ordered memory accesses. And our compiler is shown to be able to remove redundant atomic memory accesses, which the current LLVM does not. And we evaluated our compiler using several concurrent applications. We have not yet found a significant performance gain after building these applications using our compiler. The reason could be that these concurrent benchmarks do not contain the patterns our compiler optimizes. ...
Master thesis (2023) - C. Zhu, Y. Chen, Z. Erkin, Z. Al-Ars, S. Roos
Federated Learning is highly susceptible to backdoor and targeted attacks as participants can manipulate their data and models locally without any oversight on whether they follow the correct process. There are a number of server-side defenses that mitigate the attacks by modifying or rejecting local updates submitted by clients. However, we find that bursty adversarial patterns with a high variance in the number of malicious clients can circumvent the existing defenses. We propose a client-self defense, LeadFL, that is combined with existing server-side defenses to thwart backdoor and targeted attacks. The core idea of LeadFL is a novel regularization term in local model training such that the Hessian matrix of local gradients is nullified. We provide the convergence analysis of LeadFL and its robustness guarantee in terms of certified radius. Our empirical evaluation shows that LeadFL is able to mitigate bursty adversarial patterns for both iid and non-iid data distributions. It frequently reduces the backdoor accuracy from more than 75% for state-of-the-art defenses to less than 10% while its impact on the main task accuracy is always less than for other client-side defenses. ...
Genomics has revolutionized our understanding of evolution, hereditary diseases, and more. The advent of long-read DNA sequencers i.e. Oxford Nanopore Technologies' innovations, has opened many new research potentials in genomics. These sequencers produce significantly longer DNA reads, facilitating novel applications. However, this technological leap brings challenges, particularly in accurate basecalling which is the process of converting raw sequenced measurements into digital base pair sequences. While advances in basecalling accuracy have been steadily improving over the years, the computational intensity remains a bottleneck in genomic analysis workflows, demanding costly high-end GPUs for probabilistic neural network models.

The main problem this thesis addresses is the implementation of an accelerated hardware solution for the compute-intensive process of basecalling long-read sequences. The thesis presents an FPGA-based implementation of the computationally demanding Long Short-Term Memory (LSTM) layers within the basecalling network known as Bonito. However, due to the lack of floating-point arithmetic units available on the FPGA, the FPGA implementation could not achieve competitive performance compared to GPUs.

While the FPGA implementation falls short of GPU performance, it serves as a possible stepping stone toward developing an ASIC solution for implementing the Bonito LSTM layers or potentially implementing the entire Bonito model. An ASIC implementation has the potential for superior performance up to 9 times faster than a GPU implementation while additionally being cost-effective. This suggests that ASICs hold promise as a future direction for accelerating long-read sequence basecalling, allowing for faster and more affordable genomics research. ...

Collaborative and Interface-Driven Data-Streaming Accelerator Design

Master thesis (2023) - C. Cromjongh, H.P. Hofstee, Z. Al-Ars, C.B. Poulsen
In spite of progress on hardware design languages, the design of high-performance hardware accelerators forces many design decisions specializing the interfaces of these accelerators in ways that complicate the understanding of the design and hinder modularity and collaboration. In response to this challenge, Tydi has been presented as an open specification for streaming dataflow designs in digital circuits, allowing designers to express how composite and variable-length data structures are transferred over streams using clear, data-centric types. Earlier efforts in providing an implementation framework for Tydi managed to generate VHDL boilerplate code for Tydi interfaces, but offered limited design value over custom solutions due to VHDL's low abstraction level. In contrast, Chisel, with its high level of abstraction and customizability offers a suitable platform to implement Tydi-based components.

In this thesis, the Tydi-Chisel library is presented along with an A-to-Z design-process description for data-streaming accelerators. A stream-interface solution is presented that offers both compatibility with Tydi in traditional HDLs and maximum utility within Chisel through two intercompatible representations. In addition, design complexity is reduced through novel utilities like stream-complexity conversion, developed to alleviate interface specification mismatches between components. Using the presented toolchain and library, the amount of code required to specify Tydi interfaces for representative use-cases can be reduced several times compared to a Verilog description, while offering increased utility.

Tydi-Chisel aims to simplify the design of data-streaming accelerators through the integration of the Tydi interface standard in Chisel, along with helper components, syntax sugar, and verification tools. In combination Chisel and Tydi help bridge the hardware-software divide, making solo-design and collaboration between designers easier. ...
Thread pools, integrated in programming languages, packages and dependencies are widely used by developers. Thread pools assume they are running alone on the system, which is not always the case. Previous research has shown that adapting thread pool size has been effective under specific conditions. In this research, scaling the thread pool with respect to CPU and network usage is examined. However, simpler metrics can achieve better results. Two solutions are provided for algorithmically scaling the thread pool, both with a different use-case in mind. Next to that an improved way to collect CPU and network metrics is provided, allowing for real-time thread-based measurements. The result of the scaling solution is improved performance, while also offering reduced energy usage. This research shows that when multiple thread pools are running on the same machine, performance and efficiency is improved. ...

Defining an IR for Composable Typed Streaming Dataflow Designs

Master thesis (2022) - M.A. Reukers, H.P. Hofstee, Z. Al-Ars, J.W. Peltenburg, T.G.R.M. van Leuken
Tydi is an open specification for streaming dataflow designs in digital circuits, allowing designers to express how composite and variable-length data structures are transferred over streams using clear, data-centric types. This provides a higher-level method for defining interfaces between components as opposed to existing bit- and byte-based interface specifications.

In this thesis, an open-source intermediate representation (IR) is introduced which allows for the declaration of Tydi's types. The IR enables creating and connecting components with Tydi Streams as interfaces, called Streamlets. It also lets backends for synthesis and simulation retain high-level information, such as documentation. Types and Streamlets can be easily reused between multiple projects, and Tydi’s streams and type hierarchy can be used to define interface contracts, which aid collaboration when designing a larger system.

The IR codifies the rules and properties established in the Tydi specification and serves to complement computation-oriented hardware design tools with a data-centric view on interfaces. To support different backends and targets, the IR is focused on expressing interfaces, and complements behavior described by hardware description languages and other IRs. Additionally, a testing syntax for the verification of inputs and outputs against abstract streams of data, and for substituting interdependent components, is presented which allows for the specification of behavior.

To demonstrate this IR, a grammar, parser, and query system have been created, and paired with a backend targeting VHDL. ...
Master thesis (2022) - L. Xu, Z. Erkin, T. Li, J.A. Pouwelse, Z. Al-Ars, Oskar van Deventer
Third-party verified credentials (e.g. passports, diplomas) are essential in our daily life. The usage of third-party verified credentials bring us convenience in authentication. The Verifiable Credential (VC) data model is a new standard proposed by the W3C association to ease the expression and verification of third-party verified credentials on the Internet. The issuance and presentation of verifiable credentials are tamper-evident and privacy-preserving by design. However, the current verifiable credential data model lacks an explicit revocation design that guarantees the secure operations of the system. The lack of a revocation mechanism significantly limits the application of verifiable credentials. This thesis studies the revocation mechanisms of existing verifiable credential implementations. The existing revocation mechanisms are either tamper-evident or privacy-preserving. None of them can achieve the two properties together. To evolve the revocation mechanism to be both tamper-evident and privacy-preserving by design, we propose a new method which combines the BBS+ signature, a cryptographic accumulator and the blockchain. Our design enables the verifier to verify the presented credential’s revocation status without compromising the credentials holders’ privacy. We implement a proof-of-concept of our revocation mechanism to show it is practical in the real world. The experimental results show that after adding our revocation mechanism, the presentation time of a five-attribute credential changes from 22.22ms to 62.11ms (+39.89ms), and the verification time changes from 13.36ms to 44.56ms (+31.86ms). Moreover, the scalability analysis shows that our revocation mechanism can satisfy the need for revocation in the real world. ...
60 million people around the world have epilepsy, which is a neurological disorder that severely impacts their day to day life negatively. Currently available methods to reduce the effects of epilepsy are either ineffective or require expensive and invasive surgery. A new method has been found that can suppress epilepsy without the need of surgery, called Transcutaneous Vagus Nerve Stimulation (t-VNS). Detecting epileptic seizures is important for this method, as the stimulation should only be used during a seizure. Traditionally, detecting epilepsy is done using scalp-Electroencephalography (EEG), which requires a controlled environment and is hard to use in day to day life. Recently, advancements have been made in ear-EEG, which allows for EEG outside a controlled environment. This study focuses on detecting epilepsy using ear-EEG. Ear-EEG was simulated using scalp-EEG channels close to the ear. After low-pass filtering and downsampling the results were obtained using features obtained from the Wavelet Transform (WT) and Fourier Transform (FT) in combination with several Machine Learning (ML) models; these being a random forest, a Support Vector Machine (SVM), and a Neural Network (NN). Furthermore PCA was also applied to the features, with a threshold of 0%, 95% and 99%. The results clearly show that using the WT outperforms features from the FT. Furthermore, out of the three models, the NN consistently has the best sensitivity for detecting seizures. The best sensitivity was achieved using WT features with a NN and a threshold of 99% for the PCA. The accuracy and sensitivity are 99.3% and 83.5% respectively, which is comparable to previous ear-EEG based research detecting epileptic seizures.
...
The recently introduced posit number system was designed as a replacement for IEEE 754 floating point, to alleviate some of its shortcomings. As the number distribution of posits is similar to the data distributions in deep neural networks (DNNs), posits offer a good alternative to fixed point numbers in DNNs: using posits can result in high inference accuracy while using low precision numbers. The number accuracy is most important for the first and last network layers to achieve good performance. For this reason, these are often computed using larger precision fixed point numbers compared to the hidden network layers. Instead, these can be computed using low precision posit, to reduce the memory access energy consumption and the required memory bandwidth. The hidden layer computation can still be performed using cheaper fixed point numbers.
An inference accuracy analysis is performed to quantify what the effect of this approach is on the VGG16 network for the ImageNet image classification task. Using 8 bit posit for the first and last network layer instead of 16 bit fixed point is shown to result in a top-5 accuracy degradation of only 0.24%. The hidden layers are computed using 8 bit fixed point in both cases.
The design of a parameterized systolic array accelerator performing exact accumulation is proposed that can be used in a scale-out system along with fixed point systolic array tiles. To increase hardware utilization, a hybrid posit decoder is designed to enable fixed point computation on the posit hardware. Using this hardware, the entire network can be computed using 8 bit data, instead of using 16 bits for some layers. This reduces energy consumption and the complexity of the memory hierarchy. ...

Training Towards Brain Inspired Temporal Sparsity for Energy Efficient Deep Neural Networks

Master thesis (2021) - P. Preetha Vijayan, T.G.R.M. van Leuken, Z. Al-Ars, Amirreza Yousefzadeh, Manolis Sifalakis
In the recent past, real-time video processing using state-of-the-art deep neural networks (DNN) has achieved human-like accuracy but at the cost of high energy consumption, making them infeasible for edge device deployment. The energy consumed by running DNNs on hardware accelerators is dominated by the number of memory read/writes and multiplyaccumulate (MAC) operations required. As a potential solution, this work explores the role of activation sparsity in efficient DNN inference. As the predominant operation in DNNs is matrix-vector multiplication of weights with activations, skipping operations and memory
fetches where (at least) one of them is zero can make inference more energy efficient. Although spatial sparsification of activations is researched extensively, introducing and exploiting temporal sparsity is much less explored in DNN literature. This work presents a new DNN layer (called temporal delta layer) whose primary objective is to induce temporal activation sparsity during training. The temporal delta layer promotes activation sparsity by performing delta operation facilitated by activation quantization and l1 norm based penalty to the cost function. During inference, the resulting model acts as a conventional quantized
DNN with high temporal activation sparsity. The new layer was incorporated as a part of the standard ResNet50 architecture to be trained and tested on the popular human action recognition dataset (UCF101). The method caused 2x improvement in activation sparsity, with 5% accuracy loss. ...
Master thesis (2021) - R.M. Prozée, T.G.R.M. van Leuken, S.S. Kumar, Amir Zjajo, Z. Al-Ars
The development of the Spiking Neural Network (SNN) offers great potential in combination with new types of event-based sensors, by exploiting the embedded temporal information. When combined with dedicated neuromorphic hardware it enables ultra-low power solutions and local on-chip learning. This work implements and presents a viable architecture and training methodology to detect and classify audio data using Spiking Neural Networks. The architecture consists of two core components: the first component is an auditory front-end that performs low-level feature extraction. The second component is the SNN classifier supported by the spike encoder and decoder. The results show that the encoder has a major impact on the overall performance of the network. The temporal-based network is trained with help of common training methods, both supervised and unsupervised. The performance of the network is validated under both clean and different levels of noisy conditions. The impact on classification performance is analyzed and compared with traditional non-spiking Artificial Neural Networks. This in terms of classification accuracy, estimate energy consumption, and latency of inference. The proposed architectures achieve a max accuracy of 97.0% under ideal conditions. This is comparable to other non-spiking artificial neural networks, which require significantly more energy for inference. The implementation demonstrates that the architecture is a viable solution for detecting and classifying audio data. ...
Master thesis (2021) - Johann Meyer, C.C. de Visser, Z. Al-Ars, M. Mulder
Aircraft are complex systems with, in some cases, high-dimensional nonlinear interactions between control surfaces. When a failure occurs, adaptive flight control methods can be utilised to stabilise and make the aircraft controllable. Adaptive flight control methods, however, require accurate aerodynamic models - where first-order continuity is necessary for estimating the control derivatives and mitigating chattering that can reduce the longevity of components. Additionally, high-dimensional offline model identification with current approaches can take several hours for a few dimensions and this means model iterating and hyper-parameter tuning is often not feasible. Current approaches to smooth high-dimensional functional approximation are not scalable, require global communication between iteration steps, and are ill-conditioned in higher dimensions. This research develops the Distributed Asynchronous B-spline (DAB) algorithm that is more robust to ill-conditioning, due to low data coverage, by using first-order methods with acceleration and weighted constraint application. This algorithm is also suitable for continuous state-spaces. Smooth aerodynamic models can be determined in exactly n·r iterations, where r is the number of continuity equations in a single dimension and n is the number of dimensions. Moreover, memory reorganisation is proposed to avoid false sharing and conflict-free use of shared memory on the GPU to ensure that the algorithm runs efficiently in parallel. ...

Leveraging the Arrow in-memory columnar format to increase Spark efficiency in RDD computations

The ever-increasing amount of data being generated worldwide, combined with the business advantages for companies in quickly and efficiently processing such data, have accelerated the research and development into big data analytics. Existing solutions for storing and computing data can no longer give the required processing performance, and technical advancements in I/O operations and networking have further increased the distance between the requirements and the provided solutions. A number of changes, such as the introduction of new memory representations, have been proposed to reduce the overhead of existing data analytics frameworks, but they still have not been fully integrated with such frameworks for a lack of market traction.Apache Spark is among the most widely used frameworks for big data analytics, as it provides, among other functionalities, an extensive and easy-to-use API that allows to perform computations on amultitude of processing workloads.At the core of Spark sits the Resilient Distributed Dataset, an abstraction upon which other abstraction or workload-specific libraries have been built over the years. While the general development effortsfocused on creating new abstractions on top of the core library, not enough attention has been put into solving intrinsic issues of the RDD’s.Thereby, this work proposes an integration between Apache Spark and the Apache Arrowin-memory data format, that can be leveraged to solve memory- and computational bottlenecks. Such integration is beneficial for a number of reasons, such as the reduction of serialization and storage overheads, thanks to an efficient columnar-oriented data format shared among several machines in a cluster. This work proves the feasibility of such approach, introducing a number of changes in Spark and Arrow that result in a proof of concept implementation. Furthermore, it proves that such integration can be performed without the need for expensive and disruptive changes to the existing API’s, thus allowing existing workloads to be fully compatible withnew, Arrow-based techniques. The performance advantages have been evaluated, resulting in an execution time speedup of approximately 14%, with the biggest improvement being 50% reduction in execution time for wide transformations. In addition, it allows for functionalities typically executed within Spark to be offloaded to Arrow, introducing further performance improvements, with a 20% reduction in execution timefor offloaded functionalities, compared to pure Spark. ...
We consider a system of IoT nodes powered completely by energy harvesting.This work focuses on achieving the time correlation of data measurements ina network of energy harvesting sensor nodes. Time correlation is achieved byhopping a message through the whole network. This message wakes up all thenodes and lets them perform a measurement. Measurement data is added tothe transmitted message and is collected by a gateway at the end. The nodesharvest energy from a Radio Frequency (RF) source and store it in a capacitor.When the capacitor has sufficient energy, the nodes can turn on their system.A low power Wake-Up Receiver (WURx) is turned on and the nodes fall asleepwhile waiting for the incoming request message. Communication is done usingactive transmissions and the ultra-low-power WURx for data reception. Nodesconsume a continuous power of less than 2 µW in sleep mode while the WURx isturned on. The receiving sensitivity is -40 dBm, which limits the communicationrange. The request message hops through the network to overcome distancelimitation. Collisions are avoided with Clear Channel Assessment (CCA) usingthe WURx. The hidden node problem is overcome by toggling an operationalamplifier during CCA. Distance limitation is overcome by a novel network layeralgorithm. The network layer algorithm finds a directed acyclic graph (DAG)based on all nodes, starting in a single special source node and ending in agateway. Data from all the nodes are gathered in a round, where each node cantransmit one message around. The timing interval between the data collection ischosen to be bigger than the required energy divided by the minimal harvestedpower. In this way, all nodes will have sufficient energy in every time interval.The found DAG represents all important links where the nodes should wait forbefore measuring and transmitting. Other data from previous nodes are addedto the transmission of the nodes. In this way, the gateway will receive datafrom all nodes with the minimal time difference between their measurements.Simulations show that a correct gateway oriented DAG solution is always foundfor random networks. In > 92 % of the cases, all nodes are taken into account inthis solution and in 6% of the cases, just one node is missing. Nodes have beendesigned and evaluated. We can power the nodes with a minimal RF input of-15 dBm. The receiving range is found to be 8 m from a 10 dBm On-Off Keying(OOK) transmission. With 6 µW harvested energy, data from all the nodes canbe gathered every 15 minutes. ...