The rediscovered interest in space exploration has led to plans to establish outposts on the Moon and beyond. The lunar bases currently planned are to be manned incrementally, with robots performing most work. With new trends in robotics, the use of collaborating swarms has become more abundant. To support lunar operation, a distinct area within swarming, namely foraging, is proposed. Foraging systems have the primary objective of recovering resources. From literature, no foraging framework was found that included system maintenance in its mechanisms. This report aims to answer the question if this is possible while maintaining the benefits of foraging.

The research considers the fundamental act of recharging as its required maintenance task. To evaluate it dynamically, a rudimentary energetics model is included. For the framework of foraging, the work of [Adams] is used as a baseline. The newly proposed system implements an additional recharging region and role to perform recharging activities, both having major implications for role selection and agent operation. Furthermore, to enable navigation based on energy considerations, the experience communicated by mobile agents is amended to include the energy cost of a travelled path. In doing so, additional quality indicators of paths are available making path optimization a more dynamic process resulting in finer population behaviour. Finally, the decaying of beacons is updated and a fallback feature is introduced to maximize agent utilization.

The newly developed foraging system is evaluated using data collected through simulation in Webots. Simulation scenarios included obstacles with impenetrable boundaries and surfaces with increased rolling friction to emulate cost-expensive regions. Qualitative analysis identified all features of the foraging system as expected, both in the exploration and exploitation phase. Quantitative results proved that the system is able to function with the added requirements of recharging, perform path optimization with the additional path quality indicator, and can do so in various types of scenarios. With this, the research statement that foraging functionality is achievable with the practical considerations of robotics is confirmed to hold.

The effects of climate change put increasing strain on rainforests and their inhabitants, highlighting the demand for technological developments to aid in biodiversity monitoring and conservation efforts. This research work proposes a novel frame work for the autonomous placement of acoustic sensor networks on top of the rainforest canopy, using a quadrotor platform. In order to tackle the challenge posed by multirotors’ limited autonomy, an initial canopy exploration mission is considered and an exploration planner is developed to detect suitable locations on top of the rainforest canopy for sensor node deployment. Specifically, green detection and pointcloud projection is performed online and combined with our proposed sampling method to accomplish targeted exploration, towards the estimated location of detected green components. Flight experiments are conducted across multiple scenarios that mimic distinct rainforest features. The results validate the system architecture and demonstrate the effectiveness of our smart sampling method, laying the foundation for future autonomous sensor network exploration missions at larger-scale.

Genomics has revolutionized medicine and biological research by providing deeper insights into the genetic makeup of organisms, advancing our understanding of diseases, and enabling personalized medicine. These breakthroughs are driven by advancements in genome sequencing technologies, bioinformatics, and the aid of neural networks. The advent of third-generation sequencing technol ogy has further accelerated progress by allowing for long-read sequencing, which enhances the ac curacy and efficiency of genome assembly. Oxford Nanopore Technologies (ONT) offers advanced sequencers that use nanopore-based technology to read DNA sequences in real-time. However, the raw sequenced data contains noise and requires a basecalling stage to read DNA sequences with the required accuracy. Basecalling relies on deep neural networks (DNNs) to achieve high-accuracy reads, but the significant computational power required makes basecalling a costly process, especially in real-time applications. Current hardware accelerators used for these compute-intensive basecallers are state-of-the-art GPUs that cost over $10,000 per unit. This thesis explores the design of a custom hardware accelerator for ONT’s basecalling program, Bonito, which aims to provide a cost-effective alternative to existing accelerators such as Nvidia GPUs and Groq Tensor Streaming Processors (TSPs). Bonito’s DNN is dominated by five Long Short-Term Memory(LSTM)layers, accounting for 90% of its execution time. The custom accelerator targets these compute-intensive LSTMs to reduce execution time. This work provides a comprehensive analysis of LSTM performance and behavior on GPUs and Groq TSPs. It emphasizes the architectural benefits and limitations in the context of basecalling. Furthermore, it also evaluates an Application-Specific Integrated Circuit (ASIC) implementation of an existing FPGA-based LSTM accelerator design. TheanalysisshowsthatBonito’sHigh-Accuracymodel(HAC)LSTMlayerscontainmanysequential matrix multiplications that are compute-intensive and require high memory bandwidth to accommodate the data transfers. Furthermore, Bonito’s small problem size, with 384 features per vector input, causes GPUs to not fully utilize available compute cores. Combined with slow per-core performance, GPUs executing LSTMs achieve only 13.5% of the maximum TFLOP/s with FP16 precision. Groq uses a heterogeneous architecture with fast separate MXM units executing matrix multiplica tions, and VXM units executing point-wise operations. Data travels between these different compute units through streaming channels and MEM units. The LSTM analysis on Groq showed three main issues. First, Groq uses a 320-element wide data channel to transfer data across the chip, whereas Bonito has 384 hidden features as input. This leads to the 384-element input being sliced in two 192 element partial inputs as Groq supports physical tensors up to 320-element long, effectively doubling the cycle cost by using two slices instead of one. Second, performing matrix multiplications requires these slices to be transferred from MEM to MXM, by executing reload operations to the MXM weight buffer. This data transfer consumes the bandwidth on the streaming channel, which introduces stalls in the pipeline and clock cycles are spent on MEM operations, instead of compute operations in MXM or VXM. Lastly, the analysis shows that the VXM forms a bottleneck in the LSTM execution, accounting for 50% of the clock cycles, whereas the MXM accounts for the other 50%. This shows that the special functions and additions inside an LSTM cell are slowing down Groq’s overall performance. After the existing architecture analysis, the ASIC evaluation showed a synthesis result, where one block of 384 LSTM Processing Engines (PEs) achieves a clock speed of 434MHz and costs 8.07𝑚𝑚2 on a 40nmprocess node. Putting these PEs on a Groq-based chip layout of 725𝑚𝑚2 in area size, the 40nm-based PEs can achieve 79.3 TFLOP/s at FP16 precision. By correcting the 40nm process node to Groq’s 14nm process node, the 14nm-based PEs achieved 448 TFLOP/s at FP16. These results suggest that a custom LSTM accelerator could compete in performance with state-of-the-art solutions while being more cost-effective. Future work is suggested to investigate a cycle reduction in the multiply-accumulate stage and to evaluate the ASIC design using a modern process node technology, as the current ASIC design uses a 2008-based 40nm process node. This work helps future development in further optimizing a custom LSTM accelerator specified for Bonito’s DNN requirements, paving the road toward more affordable genome sequencing

Situational awareness within port areas is crucial to avoid collisions, navigate efficiently and reduce congestion. Maritime-traffic controllers constantly monitor the situation in the port and intervene when needed. This study proposes a deep learning model that predicts future vessel positions to assist in this process. The model employs a target conditioned trajectory prediction component composed of two recurrent neural networks arranged in an encoder-decoder structure that utilizes historical data points to forecast future trajectories. The model considers multiple factors, including vessel speed, location, length, depth, draught, and the tide. Additionally, this study addresses the prediction of swing manoeuvres, which are special U-turn-like manoeuvres executed during terminal arrival or departure. These manoeuvres can block a significant portion of the waterway and, as such, are essential to consider when gaining a complete understanding of future situations within the port. An integration of both models is applied to a use case study in a scoped area of the Port of Antwerp-Bruges. The models were
trained using AIS and VTS data collected at 30-second intervals. Swing manoeuvres are predicted with an accuracy of 84%, the locations of these manoeuvres are predicted with an average deviation of 212 meter and the duration error is 1.6 minutes on average. The complete predicted trajectories, including potential swing manoeuvres, have an average displacement error and final displacement error of 147 and 117 meter on average, respectively. Overall, the study demonstrates the potential of deep learning models for improving situational awareness within port areas and assisting traffic controllers in making
informed decisions.

Over the past decade, spatial audio awareness evolved into an in-demand feature in audio entertainment. The addition of sound source locations to, for instance, movies or music adds a level of auditory envelopment and spatial awareness to the audio experience. Expensive setups present in, for instance, cinema's, are able to create this envelopment by means of a large set of loudspeakers with which the desired sound fields are created. Creating this spatial envelopment in practical consumer living rooms or home cinema setups proves to be a more challenging task due to the impractical amount of loudspeakers required. To create the audio envelopment with a small amount of loudspeakers, crosstalk cancellation can be used. Crosstalk cancellation as posed in literature is, however, not robust enough to be used in practical appliances. The main cause of these bad characteristics is the objective cost function it optimizes which results in an ill-posed problem. In this thesis, the crosstalk cancellation problem is relaxed by aiming for perceptually sufficient results instead of aiming for objectively optimal results. The human auditory system has its limitations in both the perception of audio and localization of audio sources. Exploiting the limitations of the auditory system generates mathematical freedom that can be used to construct a more robust and stable crosstalk cancellation algorithm. This thesis provides the first steps towards the incorporation of audio perception in the domain of crosstalk cancellation and audio envelopment with a limited amount of loudspeakers.

In multi-agent systems reaching consensus has been a long-standing problem. A considerable amount of research has been focused on how event-triggered consensus can be used to limit the energy consumption of the communication system while still ensuring convergence to the neighbourhood of a point or a formation. These algorithms rely on having knowledge of the global state information of each agent in the network. Given this information, it is possible to consider using distributed algorithms from the field of Computer Science to achieve the same.

This thesis presents a study on average consensus algorithms for single-integrator multi-agent systems. The focus is on comparing the time required to reach convergence and the energy required by the communication system. To perform this study, three methods are used to compare the algorithms. The first method involves deriving mathematical bounds on the convergence time and the number of transmissions. The second involves a simple simulation to verify the mathematical bounds. Finally, the algorithms are compared in a practical application where robots are subjected to noisy measurements and need to assemble in a formation.

The results of the comparisons show that distributed algorithms are capable of fast convergence and require less energy for communication. Therefore, it is worth considering the use of distributed algorithms when agents have enough memory to temporarily store the global state of all agents in the network. There might still be situations in which the ETC algorithm converges slightly faster but this benefit is overshadowed by the increase in energy consumption.

The unique six-legged swarming rover Lunar Zebro is designed and produced by students from the Delft University of Technology. The objective of the rover is to accomplish an autonomous mission on the Lunar surface by 2024. This thesis evaluates a path planning algorithm that is designed for autonomous navigation in the Lunar environment. The thesis studies existing path planning algorithms and determines the essential functionalities of the algorithm and the unique requirements of Lunar Zebro. It is found that an Artificial Potential Field based path planning algorithm accommodates the determined needs and requirements.
With the help of the Artificial Potential Field path planning algorithm and the unique requirements, a vector field based algorithm is developed. The algorithm uses an attractive vector field to attract the rover to the determined target. Meanwhile, obstacles or other obstructions are denoted by a repulsive rotational vector field around the edge of the obstacles. This rotational repulsive force ensures obstacle avoidance and prevention of the local minimum trap, which often occurs in Artificial Potential Field path planning. Improvements are suggested to increase reachability and decrease path length and planning time of the rotational vector field algorithm. In the Python developed simulation, the improved algorithm accomplishes a 62% reduction in planning time compared with the original Artificial Potential Field algorithm and achieves similar path length results. Moreover, the proposed algorithm has a reachability of 90% where the Artificial Potential Field algorithm just reaches a success rate of 55%.
The thesis concludes with the future work recommendations for a low level implementation in C or either C++ to facilitate the deployment in a microcontroller.

Powered by technological advances, commercial drones have wide applications in areas such as photography, search, and rescue. However, the popularity of drones has raised technical and safety challenges for drone management and supervision. Traditional detect-before-track sensing approaches have reduced performance due to drones' low radar cross section, low velocity, and high maneuverability, and a long integration time is required to perform the spectrogram-based drone characteristics estimation.
In this thesis, a coherent electromagnetic scattering model of the drone is applied to the track-before-detect algorithm to provide a better detection performance in low signal-to-noise ratio cases and jointly estimate the dynamic state of the drone, including range, velocity, rotation frequency, and signal intensity from drone body and rotors. With the help of tracking results, a fusion of spectrogram-based characteristics estimation approaches is developed to estimate the constructional parameters of the drone, and a novel model-based number of rotor and multi-rotation frequency estimation method is proposed. The algorithms are first verified with simulation data, achieving 85%-95% detection probability at the SNR level below 5 dB and an estimation accuracy up to 96% in the number of rotor estimation. The algorithms are also validated with the experimental data, achieving agreement with the estimation results.

Unmanned underwater vehicles benefit from communication with a high data rate on relatively small distances (under 100 m). Existing communication methods are not able to provide this or present other shortcomings. Therefore, this bachelor end project focuses on a new type of underwater communication, quasi-static electric field communication. The software and modulation techniques in such a communication system are covered in this thesis. This includes a detailed analysis of modulation techniques, and especially of differential methods such as pi/M-DPSK, which can be demodulated non-coherently. Our system implements OFDM and according to simulations, it is able to achieve data rates of up to 1 Mbit/s. Additionally, this research focuses on the effect of error correction coding on the performance of the system. Moreover, an adaptive data rate control system is designed. The efficiency of the system is optimized by a power distribution algorithm. Finally, suggestions are given for a communication protocol.

The heart is one of our vital organs. It functions by periodically contracting in a rhythmic way. Sometimes, this rhythmic behaviour is affected by abnormalities in the tissue. These conditions are referred to as cardiac arrhythmias. One kind is of specific interest, called atrial fibrillation (AF). To further study AF, methods have been developed to estimate the conductivity of cardiac cells based on the measured electrical signals. In addition, other parameters of interest besides the conductivity are involved. The goal of this project was to consider one of the parameters in the electrogram (EGM) model: the sensor-to-cell height.
First, we studied the effect of the height as a parameter in the model when used for conductivity estimation. To that end, a detector was built with which we can explain the effect of all involved parameters on the ability to accurately estimate any parameters of interest.
In addition, we considered the case where the height is unknown and is estimated, thus possibly including estimation errors. The focus was on the consequences of making errors in the estimation of the height with respect to conduction block detection and conductivity estimation.
Lastly, the effort was made to estimate the height. Here, the optimisation problem of height estimation was formalised and derived as its implementable form. At first, a simplified EGM model was assumed in order to mimic and estimate cell-specific effects, i.e. the cell conductivities. Then, the height was estimated in various situations to study its behaviour and performance under different conditions. Then, also the standard EGM model was used in the same way, after which we also tested the performance of the designed algorithm in combination with existing conductivity estimation methods.

As electromagnetic waves cannot propagate sufficiently far in water, underwater communication is mainly performed using either acoustics or optics. However, none of these technologies have been proven to be completely effective in every situation. Therefore, research in new underwater communication systems can still bring large benefits to a wide range of different underwater technologies.
During this project, a promising new type of underwater communication system based on electric fields has been investigated. While two subgroups have been working on the characterization of the communication channel and different modulation techniques, this thesis focuses on the hardware needed for optimal communication. Moreover, this hardware includes both a low-noise receiver and a high power transmitter. An analysis of different design options, the detailed design of one of these options and the validation of the design are given in this report. However, to get a complete overview of the designed communication system and its performance, it is recommended to also read the other two thesis reports.

For my Master’s thesis, I developed and trained an audio-based localization system for indoor localization called AudioLocNet. AudioLocNet is based on convolutional neural networks and maps recordings from a small(10cm diameter) microphone array to a grid of locations around said array. AudioLocNet was made to be used by swarms of small robots to locate each other using audio signals. AudioLocNet was trained using orthogonal chirp signals which have a low cross-correlation. Said signals can also be used for simultaneous communications between multiple robots. These signals were recorded in indoor environments ranging from simple line-of-sight environments to reverberant non-line-of-sight ones. Audio signals are used since they form a propagational middle class when compared to radio frequency (RF) and light-based signals for localization. Whereas light requires a line of sight, audio can bend around corners; and whereas RF signals pass through walls, reaching robots that are outside of each other’s spheres of influence, audio will not.
AudioLocNet reaches high accuracies for both a coarse grid (99.96 %) and a fine grid (99.89 %) of possible locations, where only the final layer of the network architecture must be changed to account for the increased resolution of the fine grid.

This thesis describes the design and implementation of µLightDigit, which is the second iteration of the LightDigit project: a contactless air-writing system based on ambient light detection with embedded deep learning using only simple photodiodes. The system is able to classify digits 0–9 written in the air by detecting the dynamic hand shadow using a 3×3 array of simple photodiodes. A novel TinyML system is developed by transforming LightDigit from a Raspberry Pi 4 single- board computer to an STM32H743 microcontroller. This transition is burdened with a significant reduction in computation power while having the goal of creating a more robust and adaptable system. To achieve this, a new photodiode array is designed to overcome the saturation issues of the photodiode array developed for LightDigit. A preliminary analysis is performed to investigate the performance of three deep learning models on the microcontroller: a tiny convolution neural network (CNN), a tiny long short-term memory (LSTM) model, and a tiny Conv-LSTM hybrid. For the final system evaluation, the tiny CNN and ConvLSTM models are chosen which provided the best performance in the preliminary evaluation. In order to test the system’s robustness, three different indoor setups and one outdoor setup are created. These setups are created to be reproducible and serve for the verification of the system in different ambient lighting conditions and light intensities. Shadow patterns are recorded for each of the setups and are compared to explain the digit recognition results. The tiny CNN model results in an average accuracy of 84.6% with a maximum of 97.0% and a minimum of 77.8% accuracy across the four different test setups. The tiny ConvLSTM model achieves an average accuracy of 82.4%, with a maximum of 89.3% and a minimum of 72.4%. The tiny ConvLSTM model outperforms the CNN model in the outdoor setup but is generally outclassed by the CNN in all indoor setups. The CNN model is the lightest model with 792 ms inference time and a total size of 138k parameters, which translates to 147 kB. The tiny ConvLSTM model is larger with 446k parameters at 494 kB and 947 ms inference time. For the real-time inference time on the STM32H743 microcontroller, both the tiny CNN and ConvLSTM models remain below 1 second, which is fast enough for real-time applications.

Unmanned Aerial Vehicles (UAVs), commonly referred to as drones have gained increasing popularity with current technological breakthroughs. Recent reports indicate the number of registered drones in the United States have crossed 850,000 and is expected to increase multi-fold over the coming years. The widespread applications of drones include agriculture, transportation, mining, media, entertainment, etc. While drones are used for many benevolent purposes, there are also multiple real-life incidents, where drones have caused serious mishaps. Radars with high resolution are increasingly used for drone detection and classification, thanks to their long-range, all-weather monitoring capabilities. Several techniques for binary classification of drone vs no drone, drone vs birds, and different models of drones have been proposed based on the relevant features extracted from the micro-Doppler signatures or from tracks’ information. Recently, research focusing on the problem of classifying drone(s) carrying payloads has garnered considerable attention.

In this thesis, the ability of a fully polarimetric radar and a single polarimetric radar to discriminate between payloads carried by UAVs is demonstrated. A novel approach has been employed in the feature extraction algorithm, where features from individual and combined polarimetric channels are extracted for classification. Decision and ensemble fusions on the respective extracted features proved to enhance the classification performance. The robustness of the algorithm is validated on two experimental radar datasets acquired in the scenarios where the UAVs carrying payloads of different weights are hovering, flying back and forth, and flying along rectangular waypoints. Initial results for the fusion techniques provide approximately 95%-99% classification accuracy for the polarimetric and statistical features.

In the past years, small Earth Observation (EO) satellites have become increasingly capable of taking high-resolution images at high sample rates. These images contain valuable information for different sectors, such as the agricultural and military sector. Furthermore they can contain important information about the climate and climate change. Sending these images to earth requires a large amount of down-link bandwidth. This results in heavy, large power modules and communication modules, resulting in larger, more expensive (in terms of launch cost as well as in terms of production cost) satellites. This phenomenon already results in satellites not sending all information they gather, with examples of being able to send 2 minutes worth of data per orbit (approx. 90 minutes) not being out of the ordinary. As more and more satellites are transmitting data towards earth the communication is also expected to become even more power-intensive (or even more limited), since the (theoretically) available bandwidth per satellite is reduced. Therefore a shift towards a different approach is necessary. Smarter ways to get the relevant information to earth have to be developed. In contrast with the "common knowledge" that is often applied in the field of object detection, using the highest possible image quality does not transfer to the best trained network when gathering a dataset of satellite images, since the main constraint is the bandwidth available for transmitting images, where "normally" the largest constraint is the amount of man-hours spent on annotation. Compression of training images with JPEG-XR quality level 2 during the gathering of training images results in a better "bandwidth-efficiency", in the dataset used for this research at least up to 30000 images. It was also found that when more bandwidth is available and thus more images can be added to the training set, the optimal amount of compression tends to decrease. This results also lies in line with the result that the "final accuracy" (the predicted accuracy of a model trained on an infinite amount of images) of the models tend to improve with better image quality. From this it can be concluded that for the optimal approach the training images should be compressed as far as possible at the start of training, to then decrease the amount of compression as the mission progresses and more cumulative bandwidth is available.

Advanced automotive vehicles are based on the real-time fusion of an increasing number of automotive sensors. For precise fusion of different sensors, measurements need to be synchronized both temporally and spatially. This thesis aims to design a hardware temporal synchronization block as part of the PRISTINE systolic array accelerator project for multi-sensor data fusion. In this process, we study and address several temporal sensor synchronization issues that are characteristic of the considered system as well as any other typical sensor fusion system. First and foremost, we handle the problem of estimating the actual time of sensor measurement by exploring well-known filtering techniques such as Kalman, mean and median filters. A suitable filter is selected for implementation based on the statistical characteristics of the observed sensor cycle times, the complexity of the filters and the quality of obtained estimates. Next, we address the issue of reconstructing incoming sensor data streams according to the estimated sensor measurement times while maintaining minimal latency and synchronization error by employing an adaptive stream buffering technique utilized in distributed multimedia systems. An analysis of the effects of the stream synchronization algorithm's parameters on buffering latency and synchronization error was presented. Finally, the above synchronization solution was efficiently implemented on hardware by making certain modifications and design decisions to the algorithm. A method to evaluate the whole temporal synchronization process is proposed and the obtained results on real sensor data are presented.

Tactile internet promises to enable humans to manipulate remote physical environments. The human interacts with the remote environment by manipulating a haptic device. Recently, research effort into tactile internet and haptic devices has seen a significant increase. However replicating one aspect of executing an action, feeling textures, is still in its infancy. Current devices that can portray texture are severely limited, which also interferes with the ability to research the topic. In this thesis, we propose a novel device that can realistically portray textures to the fingertip, and respond to displacement in real-time. We call this a Haptic Texture Device (HTD). The design features custom actuators, with low latency < 0.6ms and at least 50mN per mm2 of pushing force. The actuators are placed in a grid with a 2.0mmpitch at the fingertip. TheHaptic TextureDevice (HTD) can be moved over a 2D surface. Movement of the HTD results in real-time updates to the portrayed texture to reflect the displacement. The end-to-end delay between displacement and update of the texture was measured to be 17 ms. However, we demonstrated that the theoretical limit of the end-to-end delay could bewell below1 ms, by making use of higher quality displacement sensor. Readily available heightmaps are used as a source of texture data. Because a method for characterizing the performance hitherto did not exist, a novel benchmark was developed that incorporates realistic textures and human participants. The results of our benchmarking yielded an average score of 61%. Since the device is the first of its kind, it cannot be compared to similar devices yet, but it can be used as a starting point on which future HTDs can improve.

People counting data in offices is used in many applications like HVAC system control and space management to increase comfort, decrease energy consumption and optimise space utilisation. In contrast to past approaches using imaging modalities that tend to be either expensive or intrusive, we consider single-pixel thermopile sensors for people counting. These sensors may already be deployed as part of a smart lighting system to provide temperature data for HVAC controls. Firstly, a statistical sensor model for thermopile temperature measurements is proposed. The proposed people counting method enhances the CUSUM RLS algorithm to estimate temperature change caused by people entering or leaving. We estimate mean temperature changes upon detection of an occupancy event, and then estimate the people count using a maximum likelihood on the estimated temperature change. Finally, PIR vacancy data is merged with the people count estimation to increase accuracy. We obtain an average counting error of 0.11 and 0.19 for 90% of the instants respectively when considering 15 minute windows for simulated and experimental datasets. A second aspect of the thesis considers the problem of commissioning plan detection. We leverage the two-sided CUSUM signals to address this problem. The two-sided CUSUM scores for a pair of sensors are used to calculate similarity measures; these features are used in a Random Forest Classifier to detect commissioning changes of the sensor pair. Using simulated data with the thermopile signal model, we show that the proposed method achieves a true positive rate (determining the correct layout) of 90.2% and false positive rate of 1.3%.