Multi-modal 3D object detectors achieve state-of-the-art performance but remain notoriously brittle to asymmetric sensor degradation, such as when LiDAR point clouds become sparse in new environments. In this paper, we investigate unsupervised cross-modal adaptation to rescue a degraded sensor using an unaffected reference modality, without requiring target-domain labels. Using UniBEV on the nuScenes dataset, we simulate severe degradation by reducing LiDAR resolution from 32 to 8 beams. We systematically compare two leading adaptation paradigms anchored by the reliable camera stream: output-level camera pseudo-labeling and feature-level cross-modal mapping via a Bird's-Eye-View (BEV) Attention U-Net. Our experiments reveal a compelling insight: while feature mapping successfully aligns coarse spatial structures (improving LiDAR-only mAP by 5.6%), it fails to preserve fine-grained localization metrics. In contrast, simple confidence-filtered pseudo-labeling provides a significantly stronger recovery, yielding a 13.1% mAP improvement. Ultimately, our findings suggest that basic feature-level alignment may be insufficient to restore fine-grained 3D detection under severe spatial degradation, indicating that direct output-level supervision can be a more effective and reliable strategy for cross-modal adaptation in this regime. ... Read more

Detecting changes in indoor environments is essential for applications such as automated inventory monitoring.We address this problem using only active acoustic sensing: a speaker emits a known signal, distributed microphones record the sound, the Room Impulse Responses (RIRs) are obtained through deconvolution, and a neural network determines which objects have been added or removed. We address two RIR-based tasks for passive objects: object count regression, which estimates the number of silent objects present in a room from multi-channel RIRs, and multi-label binary change detection, which predicts per-object presence changes between two observations. To the best of our knowledge, neither task has been previously addressed in the acoustic domain for passive objects. Both tasks are solved using a VGGish-based architecture operating on log-Mel spectrograms. To overcome the scarcity of real-world training data, we develop a simulation pipeline using SoundSpaces 2.0and Blender to generate RIRs for arbitrary room configurations.We validate our approach through four incremental experiments.First, we reproduce a localization baseline on real RIRs from the SoundCam dataset, achieving a mean error of 15.5 cm with ten microphones at 48 kHz. Second, we quantify the sim-to-real domain gap, finding that in-domain simulated performance(18.2 cm) approaches the real baseline but cross-domain transfer fails, with errors exceeding the random-guess baseline. Third,in simulation, object count regression achieves a mean error of 0.38 boxes within the training distribution, well below the random baseline of 1.23, though extrapolation to unseen counts degrades (mean error of 0.67 boxes). Fourth, the binary change detection model correctly identifies changes for nine out of ten object locations, achieving a mean per-box accuracy of 0.935,well above the 0.5 random baseline. Direct-path occlusion reduces mean accuracy from 0.935 to 0.840, with failures concentrated at spatially clustered locations. Models trained on small change events (at most three simultaneous changes) generalize meaningfully to larger change magnitudes, retaining AUC above 0.9for seven of ten locations. These results provide initial evidence for the feasibility of acoustic change detection for multiple silent objects, while highlighting the sim-to-real gap as the primary barrier to practical deployment. ... Read more

Wildfires spread in hours, ships change course in minutes, and natural disasters evolve faster than most satellites can observe. Current Earth Observation (EO) systems struggle to keep up: wide-area sensors provide coverage but little detail, while high-resolution satellites revisit too slowly.

This thesis, in collaboration with European Space Agency Φ-lab, investigates AI-based Tip and Cue, an EO strategy in which a wide field-of-view satellite (“Tip”) detects events and autonomously tasks a high-resolution satellite (“Cue”) for targeted imaging.

An end-to-end mission simulation framework was developed with three components: (1) an Orbital Simulation Framework modelling satellite dynamics, tasking, and pointing; (2) an Off-Nadir Imaging Rendering Pipeline generating off-nadir imagery with geometric and radiometric effects; and (3) an Onboard Detection Module deploying the transformer-based DEIMv2 model. A benchmark was developed and applied to whale detection.

Results show a two-satellite Tip-and-Cue system can approach the performance of a 16-satellite Walker constellation, achieving comparable detection rates, latency, and coverage with an order of magnitude fewer satellites. ... Read more

Deep neural networks are increasingly used for ego-motion estimation. Often, in self-supervised ego-motion networks, it is decoded from a depth network and until now has not been decoded from an optical flow network. This is surprising given the tight relationship between optical flow and ego-motion. While both representations are widely used in learning-based approaches, the extent to which their latent space encodes motion information remains poorly understood. This paper presents a controlled analysis of how well depth-based and optical flow-based neural networks encode ego-motion. Using supervised depth, flow and pose networks trained on TartanAirV2, we probe motion information by attaching identical minimal pose decoders to frozen encoders. Representational space analysis through Centered Kernel Analysis (CKA) and feature space analysis through Principal Component Analysis (PCA) are used to examine how motion information is structured across network hierarchies. This work shows that optical flow representations encode ego-motion information more explicitly, in a lower dimensional, linearly accessible structure as opposed to depth representations, which exhibit weak alignment with pose. These findings suggest that in a self-supervised setting, ego-motion estimation can best be decoded from an optical flow network as opposed to a depth network. ... Read more

Automated analysis and interpretation of 3D urban environments from laser-scanned point clouds has emerged as a critical research area with broad applications in urban planning, land administration, autonomous driving, and navigation. Despite remarkable progress in this field, researchers face two key challenges: (i) the comparatively slower advancement of methodologies for 3D point cloud analysis compared to 2D image-based techniques, and (ii) the difficulty of scaling these methods to large and complex real-world urban environments. This thesis addresses both aspects by exploring methodological innovations in 3D point cloud processing and investigating their applicability to large-scale urban settings, with an overall aim of supporting more robust and reliable interpretation of 3D urban scenes.... ... Read more

This thesis addresses the problem of vision-based tracking of Vulnerable Road Users (VRUs) such as pedestrians and riders in the context of Automated Driving (AD). The work primarily builds on camera-based perception, while also considering LiDAR as a complementary sensor. Robust tracking of persons is a critical perception task, as it enables downstream modules for motion prediction and safe planning. The motivation stems from the disproportionate involvement of VRUs in traffic fatalities, as globally pedestrians and riders account for more than half of all road deaths, particularly in urban environments. By providing temporally consistent trajectories and identity preservation across occlusion and re-entries, tracking contributes to safer and more anticipatory automated driving systems.

The thesis first covers an extended review of the state-of-the-art in multi-object tracking. A modular framework is outlined to describe traditional tracking pipelines. The review then focuses on deep learning-based methods. Early methods typically extended classical pipelines with deep features, while later works moved toward end-to-end formulations and transformer-based architectures. This survey provides the methodological foundation and highlights the transition toward data-driven approaches.

As these approaches rely on large amounts of data for both training and evaluation, fully manual annotation becomes increasingly challenging and costly, as the required effort grows rapidly with dataset size. To address this challenge, the thesis develops a semi-supervised trajectory generation pipeline. The approach leverages camera detections, LiDAR-based estimation of 3D locations from 2D boxes, and vehicle inertial sensing for ego-motion compensation. Based on this data, the approach generates temporally dense pseudo ground-truth (i.e. 2D bounding boxes, 3D person locations) from sparse, manual annotations at keyframes. Two complementary strategies are employed to generate tracks within each batch, whereby each batch is defined to begin and end at a keyframe. The first is a graph-based optimization for global association that leverages a spatio-temporal graph. The second is a tracklet-based approach that combines tracklet generation with subsequent tracklet merging. For both strategies, the outputs obtained within each batch are subsequently aggregated into complete trajectories, guided by the keyframe annotations. An experimental evaluation of these approaches, including comparisons to baselines, ablation studies, and ceiling analyses, demonstrates that the tracklet-based pipeline achieves the most robust overall performance. This approach significantly reduces annotation effort while producing temporally consistent trajectories suitable for training deep tracking models.

The EuroCity Persons (ECP) 2.0 dataset is introduced as a large-scale image dataset for person detection, tracking, and prediction in traffic. The dataset was collected onboard a vehicle driving through 29 cities in 11 European countries. It contains more than 220K unique person trajectories, in more than 2.0M images and comes with a size of 11 TB. ECP2.0 is about one order of magnitude larger than previous state-of-the art person datasets in automotive context. It offers remarkable diversity in terms of geographical coverage, time of day, weather and seasons. The pipeline proposed in this thesis is used to generate the dense annotations, providing temporally consistent pseudo ground-truth on top of the recorded image data. Experiments analyze the resulting annotation quality and object-dependent effects. The benefits of using the pseudo ground-truth for training and testing of state-of-the-art tracking methods are further demonstrated, showing improved tracking accuracy and robustness compared to training on sparsely annotated or purely detector-based supervision, while also highlighting remaining challenges in handling long-term occlusions and identity switches. The ECP2.0 dataset is made freely available for non-commercial research use.

Finally, this thesis improves person appearance embeddings as a means to strengthen identity association in multi-object tracking. To this end, a novel method for creating appearance embeddings to identify individual persons using an object Re- Identification (ReID) framework is introduced. CLFormer (Camera LiDAR Transformer) is presented, a transformer-based architecture that incorporates multi-modal data from both camera and LiDAR sensors. The 3D Cuboid-Inclusive Point Embedding (3D-CIPE) is introduced, which leverages rich data from LiDAR point clouds and 3D cuboids to add a learnable embedding into the transformer structure. Additionally, through ablation studies, various strategies for the early and late fusion of multi-modal input data are explored and analyzed. To evaluate the proposed CLFormer, the nuScenes dataset [14] is reinterpreted for ReID purposes and used for the experiments. CLFormer demonstrates a significant improvement in performance, outperforming the image-only baseline with an increase of 2.3 in mean Average Precision (mAP).

In conclusion, this thesis advances the state of-the-art in VRU tracking for automated driving by providing an extended methodological review, novel semi-supervised dataset creation techniques, a large-scale benchmark dataset, and a complementary ReID method that provides stronger identity cues for tracking. Together, these contributions offer new methodological insights, practical tools, and benchmarks that can support future research and development in the field of automated driving perception and tracking. Future work may focus on further improving the semi-supervised annotation method, particularly by reducing or even moving beyond the reliance on manual keyframe annotations, toward full unsupervised or self-supervised methods. In general, the long-term vision for multi-object tracking in automated driving points toward systems that unify perception, prediction, and decision-making in end-to-end frameworks, enabling robust and reliable operation in diverse and challenging real-world environments.
... Read more

Visual Place Recognition (VPR) is a key task in computer vision and robotics, enabling loop closure in SLAM, image-based localization, landmark retrieval, and navigation. While deep-learning approaches have improved robustness to viewpoint, illumination, seasonal, and dynamic changes, three less explored challenges—domain generalization, uncertainty estimation, and localization accuracy—remain critical. This thesis demonstrates that test-time reference maps, typically used only for retrieval, can be exploited to address these challenges without additional sensors or retraining.

A unified evaluation framework, VPR-Bench, is introduced to standardize datasets, metrics, and evaluation practices across robotics and vision communities. VPR-Bench enables meta-analyses of descriptor size, runtime trade-offs, viewpoint and illumination invariance, and retrieval efficiency, highlighting that no single VPR method is universally best.

To improve cross-domain robustness, Reference-Set Finetuning (RSF) is proposed: a self-supervised finetuning strategy using test-time reference images to reduce train-test domain gaps. For reliability, Spatial Uncertainty Estimation (SUE) leverages reference map metadata to quantify the spatial spread of top-ranked poses, outperforming lightweight methods and complementing geometric verification. Finally, Continuous Place-descriptor Regression (CoPR) densifies the feature space by regressing descriptors at novel poses, reducing localization errors caused by map quantization and enhancing accuracy when combined with viewpoint-variant encoders.

Overall, this thesis reframes the reference map from a passive database to an active, exploitable resource. By systematically leveraging map information through RSF, SUE, and CoPR, it delivers measurable improvements in robustness, reliability, and localization accuracy, advancing map-aware VPR for real-world robotics and autonomous systems. ... Read more

Abstract—Crowd-sourced imagery is increasingly important for urban mapping and visual localization. However, its reliability is limited by GPS inaccuracies and heterogeneous capture condi- tions, including device variability, viewpoint differences, illumi- nation changes, and temporal shifts. In these settings, achieving metric-scale pose estimation remains a central challenge. Deep Learning-based pose estimation models address this problem by learning to estimate the 6-DoF pose using geometric cues between image views and metric supervision during training on large datasets. This encourages spatial consistency and sup- ports generalization across diverse conditions. Recent learning- based architectures, often based on vision transformer encoders, approach the task through unified multi-task frameworks that jointly predict metric depthmaps and 2D–2D correspondences, with relative pose estimated downstream. This thesis evaluates whether such frameworks predict accurate metric depthmaps under domain shifts. Experiments show that, even with scale correction through data-driven fine-tuning with metric supervi- sion, depth predictions from multi-task relative pose estimation models fail to generalize reliably to out-of-domain environments. In contrast, monocular models, trained on significantly larger and more varied datasets, demonstrate strong zero-shot reliability for metric depth prediction. A hybrid pipeline is proposed that combines the geometric consistency of relative pose models with the stable metric cues of monocular models, enabling robust pose estimation in crowd-sourced outdoor environments. ... Read more

Autonomous driving relies heavily on cameras and LiDAR for 3D perception, yet these vision-based sensors face limitations under poor illumination, adverse weather, or occlusion. Inspired by human hearing, we explore whether microphone arrays can enhance vehicle perception. We propose SonicVision, the first bird’s-eye-view (BEV) acoustic detection framework that jointly localizes and classifies traffic participants using sound alone. Our method employs a horizontally arranged 32-channel microphone array and transforms raw waveforms into short-time Fourier transform (STFT) features augmented with positional embeddings. A ResNet-based architecture is trained with novel Gaussian label representations to predict class-conditioned direction–distance distributions. To support this study, we collect three datasets (simulation, test track, and real road) with synchronized audio and LiDAR, where LiDAR detections serve as pseudo-labels. Experiments show that SonicVision significantly outperforms beamforming-based baselines, achieving accurate localization and classification performance. In some cases, our approach is able to identify objects that are missed by LiDAR, suggesting its potential as both an independent sensor and a complementary modality. These results provide the first evidence that low-cost microphone arrays can meaningfully contribute to 3D perception for autonomous vehicles. ... Read more

Autonomous driving is a rapidly growing sector that is attracting increasing attention from industry and academia. The rise of deep learning techniques has made it possible for autonomous vehicles to perceive the environment around them, including detecting objects of interest around the vehicle. The technology used to detect objects is quite dependent on sensors such as cameras, radars, and LiDARs, which are commonly used in autonomous vehicles for 3D object detection. Current state-of-the-art 3D object detection methods often use a fusion of LiDAR and camera features. As such, the perception abilities of autonomous vehicles are susceptible to sensor corruptions, whether the corruptions are caused by internal sensor faults or extreme environmental conditions. A particularly hazardous corruption occurs when a LiDAR does not record points in certain regions, which can be caused, among other reasons, by dirt accumulation on the LiDAR, a wet ground, and dark-colored objects, which can absorb LiDAR beams or reflect them away from the LiDAR. Corruptions to the LiDAR can be very dangerous as they contribute significantly to the ability of multi-modal 3D object detectors, and the robustness of multi-modal 3D object detection methods against such corruptions is understudied. In this thesis, I propose Ada-UniBEV, which detects missing laser beam returns for the LiDAR based on its geometric pattern of shooting beams, and determines a severity of corruption based on the number of missing beams. Camera features corresponding to the corrupted region of the LiDAR are weighed as more important, with the exact weight being determined as a function of the severity of corruption. The weights also vary spatially around the autonomous vehicle depending on the properties of each spatial location. Ada-UniBEV improves the 3D object detection performance over state-of-the-art UniBEV in moderate to high-severity LiDAR corruption scenarios involving missing points while maintaining the same performance on clean data. ... Read more

Visual Place Recognition (VPR) remains a challenging problem, particularly under difficult conditions such as night-time or winter weather, which are often underrepresented in existing training datasets. Although transformer-based models have recently advanced the state-of-the-art, their high computational demands can hinder deployment in real-world robotic systems. This thesis proposes a new data augmentation strategy for VPR using image-to-image Vision Foundation Model InstructPix2Pix to generate realistic visual variations such as night and snow scenes from the original training data. These synthetic augmentations are added to the original training dataset to extend dataset diversity without requiring additional data collection. To further improve performance, the method is combined with more advanced augmentations using the Kornia library, which already improves robustness over traditional augmentation techniques. Experiments on multiple benchmark datasets show that lightweight, ResNet-based models trained with our VFM augmentations achieve significantly improved performance under challenging visual conditions. Additional ablations demonstrate the importance of careful prompt design and hyperparameter tuning. Overall, this work shows that VFMs can serve as practical tools for targeted dataset augmentation, improving the robustness of existing VPR methods in difficult scenarios.
... Read more

Visual counting is an important task in computer vision with broad applications in areas such as crowd monitoring, agriculture, and environmental analysis. While deep learning has significantly advanced this field by enabling models to learn robust feature representations, deep learning approaches suffer from sensitivity to data imbalances, which occur in the distribution of object counts across counting datasets as a result of annotation effort. Most state-of-the-art counting models, categorized into clustering-, detection-, regression-, and density estimation-based methods, are built upon Convolutional Neural Networks (CNNs) and Transformers, both of which are known to be susceptible to imbalances in the training data. This study introduces a hybrid model that incorporates a programmatically guaranteed counting mechanism using the RASP language and the Tracr compiler, enabling the construction of Transformer-based models that can reliably execute predefined tasks, such as counting. By combining this exact counting mechanism with a trainable embedding module, we present a model that is capable of learning to count various tokens, even under significant data imbalance. We validate our approach on a synthetic, imbalanced dataset and compare its performance, training time, and data efficiency against standard CNN- and Transformer-based models. Results suggest that our method achieves strong generalization across the full spectrum of object counts while requiring less training data, highlighting the potential for this architecture to be further investigated and adapted to be used for robust and efficient visual counting. ... Read more

Perception is a fundamental component of autonomous and self-driving vehicles, with reliable object detection and understanding of the environment being critical for safe operation. While lidar and camera based systems are widely used, radar remains a promising option due to its robustness in poor weather conditions and ability to directly measure the radial velocity of objects via the Doppler effect. However, radar’s sparse data and resulting limitations have constrained its potential. This thesis investigates the use of dual automotive radar setups to mitigate these limitations, and use the specific advantages of such a setup to improve full velocity vector estimation methods. A novel algorithm is proposed to achieve more accurate velocity estimation in non-ideal, real world conditions. The work further explores how improved velocity estimation can be used to improve classification performance using graph neural networks. Here it was found that including velocity information via a ground truth method did increase classification performance significantly, though the same result could not be obtained via the previously mentioned velocity estimation method. To support evaluation, a simplified simulation environment and ground truth velocity data for the RadarScenes dataset are developed. This research aims to close the performance gap between radar and other more data-dense sensors, offering a robust and more cost effective alternative, especially in conditions where optical systems under perform. ... Read more

Accurate and up-to-date road maps are vital for Automated Vehicles (AVs) to navigate urban environments safely and predict the behavior of surrounding agents. However, generating such maps typically requires manual annotation or depends on expensive sensor-equipped vehicles, limiting scalability. Aerial imagery offers a more scalable alternative, but existing methods either fail to capture both geometric and topological details or do not generalize well to unseen urban areas.
This thesis addresses these challenges by introducing SAM-Maps, a novel framework that automatically generates road maps from aerial imagery without requiring additional model training. Leveraging foundation models, SAM-Maps extracts both the drivable area geometry and road connectivity of urban environments.
Experiments on the View-of-Delft Prediction dataset demonstrate that SAM-Maps achieves a recall of 43.4% in a fully automatic mode, improving to 75.6% with minimal manual steps. Additionally, trajectory prediction experiments using the state-of-the-art Wayformer model show a 37.9% improvement on the minADE6 metric when incorporating SAM-Maps, compared to scenarios without map input. To the best of our knowledge, this is the first method to extract both drivable areas and road connections for European urban environments from aerial imagery, providing a scalable solution for road map generation. ... Read more

Plastic pollution in rivers is a growing environmental issue with widespread impacts. Monitoring the movement of plastic waste across different river systems is challenging due to environmental variability and the limited availability of labeled data. This thesis investigates camera-based methods for detecting floating macroplastics in rivers and explores ways to adapt detection systems to new locations with minimal data. Collecting data from the Limmat River in Zurich and the uMhlangane River in Durban, South Africa, the study assesses the impact of domain shifts on detection performance and proposes a semi-automated annotation pipeline to improve labeling. Furthermore, it tests techniques like few-shot learning and pseudolabeling to address the performance dip. The results show that model performance decreases significantly when applied to new locations but that even with minimal data, camera-based monitoring can provide useful insights for understanding waste movement and informing plastic waste management strategies. ... Read more

Temporal segmentation of medical procedures holds the potential to improve patient safety, provide decision support to clinicians, and serve as the basis for context-aware robotic assistance systems. However, clinical adoption remains hindered by two key challenges: the scarcity of annotated data and limited generalizability across diverse surgical settings. This thesis therefore explores 2D skeleton-based temporal segmentation as a privacy-preserving and data-efficient alternative to conventional RGB-based methods. Using the CAG-skeleton dataset, which consists of pose sequences extracted from external cardiac angiography (CAG) recordings, the study investigates various model architectures and limited supervision strategies for identifying 14 procedural phases.

A two-stage framework, combining a skeleton-based feature extractor with a temporal model, was adopted. A review and comparison of proven models revealed combinations of PR-GCN or MS-G3D feature extractors with LSTM or TCN temporal models to hold the most promise in the low-data medical domain. After training all combinations on low-data subsets of the CAG-skeleton dataset, it was found that all models outperformed a non-learning baseline model, which always predicts the mean procedure. Between the learning models, clip-wise segmentation accuracy differences held no statistical significance, but LSTM-based models showed a statistically significant superior understanding of sequential order. Considering both sequential metrics and computational efficiency, the PR-GCN + LSTM combination was selected for extensive evaluation, achieving a clip-wise segmentation accuracy of 83.95\% when trained on 146 CAG procedures.

To further address the data scarcity challenge, two limited supervision approaches were explored. Transfer learning using the Kinetics-skeleton dataset showed no statistically significant performance gains, suggesting that the knowledge learned from Kinetics-skeleton does not effectively transfer to the surgical domain, and/or that the information transferred is relatively easy for the model to learn from scratch during training. In contrast, pseudo-labeling via class-balanced self-training showed great potential for reducing annotation requirements as it provided consistent improvements to the models' clip-wise segmentation accuracy in the low data regime.

Overall, this study introduces skeleton-based representation as a modality holding large potential for medical temporal segmentation and highlights pseudo-labeling as an effective strategy for reducing annotation requirements.
... Read more

Visual place recognition (VPR) is a form of visual localization. Current approaches are designed to handle common VPR challenges, such as appearance and viewpoint variations. With the introduction of DINOv2, vision foundation models have been used as feature extractors to improve performance for VPR techniques, as they show great generalizing capabilities for image representations. By fine-tuning these large models on VPR-specific datasets, performance increases even more. A problem with these big VPR datasets is the bias towards urban environments. To solve this problem, we propose to use a simple pipeline to fine-tune existing techniques on the reference databases of test datasets. Our experiments show that performance improves by reference database fine-tuning for multiple techniques on different datasets. To handle appearance and viewpoint variations as well, image augmentations can be used during training. With this complete pipeline, techniques improve performance. The experiments show improvement even if a large query-reference domain gap exists for that dataset given that a part of the test queries are know during fine-tuning. ... Read more

The training process of machine learning models for self-driving applications suffers from bottlenecks during loading and processing of LiDAR point clouds with large storage complexity.
Many studies aim to remedy this problem from an implementation perspective by developing efficient data loading and processing pipelines.
This study, on the other hand, explores an alternative approach by augmenting data representations to achieve lower storage complexity known as point cloud compression.

A broad analysis is presented on novel point cloud compression codecs using tensor decomposition methods.
Several point cloud representations and tensor decomposition methods are considered over a range of hyperparameter choices and compression values.
In order to assess the performance of the presented codecs: the compression rate, quality of the reconstruction, and time complexity is compared to the octree-based baseline model: TMC13.
Compared to the baseline model, the performance of the presented tensor decomposition-based codecs falls short. One of the presented codecs does notably outperform the others. This codec uses synthetic tensorization followed by sorting using z-location and decomposition using the TT-SVD algorithm.
Sorting by z-value isolates the ground plane, which is a dominant low-rank feature, which can effectively be decomposed using the TT-SVD algorithm yielding adequate results.

Several limitations of the presented tensor decomposition-based codecs are: the omission of bitwise compression on the factor matrices, and the trade-off between bitwise precision and truncation due to tensor decomposition.
Future work could improve in these areas along with considering the use of different heuristics and optimizing the tensor network topology. ... Read more

The ever-increasing complexity of Artificial Intelligence (AI) models has led to environmental challenges due to high computation and energy demands. This thesis explores the application of tensor decomposition methods—CP, Tucker, and TT—to improve the energy efficiency of large Convolutional Neural Networks (CNNs) during inference by reducing energy consumption. The energy consumption of several convolution layers was measured using a watt meter across various CNN configurations and different hardware architectures (Central Processing Unit(CPU) and Graphics Processing Unit (GPU)). In addition, several regression models were fitted to estimate energy savings, incorporating memory usage. It was found that TT decomposition consistently provided the most significant energy savings across various compression ratios, influenced by CNN hyperparameters such as input/output channels, feature sizes, and kernel sizes, whereas CP decomposition was the least effective in reducing energy. The GPU implementations generally resulted in additional energy consumption, and the GPU regression models suggested a need for more complex relationships. The thesis also revealed that the efficiency of tensor decompositions might be highly dependent on the implementation details of software libraries, such as TensorlyTorch, which can significantly impact the computation and memory complexities. These findings underscore the importance of both hardware specific considerations and careful software implementation in achieving energy efficient CNNs, providing a foundation for further research in energy-constrained environments. ... Read more

The shift to sustainable energy sources has increased demand for Energy Transition Metals such as nickel, copper, cobalt, and manganese. To satisfy this need while reducing the negative social and environmental effects of conventional mining, Deep-sea Nodule Collection (DSNC) appears to be a feasible option. Despite its potential, DSNC encounters difficulties with path planning since it depends on imperfect global paths, which result in inefficiencies and safety issues. This thesis looks into the possibility of using a local path planning algorithm, specifically an Interpolating Curve Planner (ICP).

To assess the ICP's performance under various model hyperparameters and environmental scenarios, a simulation tool is created. The study examines how well the ICP performs in terms of area coverage, collection rate, and reliability along different hyperparameters and presurvey measurement resolutions. It integrates novel production rate objectives, previous track following, and real-time sensor measurements within an environment generated from bathymetric data.

The examined local path planner shows a step towards optimised DSNC. Further research is needed to refine objectives and the structure of the local path planning algorithm to maximize its potential in addressing DSNC challenges effectively.
... Read more