Understanding weight-magnitude hyperparameters in training binary networks

Binary Neural Networks (BNNs) are compact and efficient by using binary weights instead of real-valued weights. Current BNNs use latent real-valued weights during training, where several training hyper-parameters are inherited from real-valued networks. The interpretation of several of these hyperparameters is based on the magnitude of the real...

Distributed Gaussian Process for Multi-agent Systems

This work is focused on the distributed system, i.e. Multi-agent Systems (MAS), with application in environmental monitoring and learning. The specific task is to develop algorithms, i.e. Gaussian Process (GP), that are robust, accurate and fully-distributed to learn the unknown spatial environmental field. The two main problems are (1). how to...

Green AI: An empirical study

In this work, we look at the intersection of Sustainable Software Engineering and AI engineering known as Green AI. AI computing is rapidly becoming more expensive, calling for a change in design philosophy. We consider both training and inference of neural networks used for image vision; to reveal energy-efficient practices in an exploratory...

Effect of parameter tuning on reducing the number of queries required to perform model stealing

Model extraction attacks are attacks which generate a substitute model of a targeted victim neural network. It is possible to perform these attacks without a preexisting dataset, but doing so requires a very high number of queries to be sent to the victim model. This is otfen in the realm of several million queries. The more difficult the...

Automated Machine Learning in Medical Image Segmentation

Semantic Segmentation of medical images are used to improve diagnosis and treatment. In recent years, the application of machine learning methods are increasingly used. However, the design of these models is difficult and time-consuming. In this thesis, we investigated the automation of this process using an Automated Machine Learning (AutoML)...

Contextual Hyperparameter Optimization for the Train Unit Shunting Problem

One of the planning problems encountered by the Dutch Railways (NS) at shunting yards is the train unit shunting problem (TUSP). This problem considers idle train units that have to be parked, be cleaned, undergo regular maintenance, and be reconfigured into the scheduled departing train compositions. A local search algorithm is being developed...

A Systematic Comparison of Search-Based Approaches for LDA Hyperparameter Tuning

Context: Latent Dirichlet Allocation (LDA) has been successfully used in the literature to extract topics from software documents and support developers in various software engineering tasks. While LDA has been mostly used with default settings, previous studies showed that default hyperparameter values generate sub-optimal topics from software...

MultiTune: Dynamic budget allocation for hyperparameter tuning

Hyperparameter optimization(HPO) forms a critical aspect for machine learning applications to attain superior performance. BOHB (Bayesian Optimization and HyperBand) is a state of the art HPO algorithm that approaches HPO in a multi-armed bandit strategy, augmented with Bayesian optimization to drive configuration sampling. However, BOHB...

A Literature Study into Hyperparameter Systematization for Deep Learning-based Side-Channel Analysis

With the recent increase in computational power, deep learning is being applied in many different fields. Deep learning has produced promising results in the field of side-channel analysis. However, the algorithms used to construct deep neural networks remain black boxes, which makes it hard to fully employ the capabilities of attacks performed...

Black Magic in Deep Learning: Understanding the role of humans in hyperparameter optimization

Deep learning is proving to be a useful tool in solving problems from various domains. Despite a rich research activity leading to numerous interesting deep learning models, recent large scale studies have shown that with hyperparameter optimization it is hard to distinguish these models based on their final performance. Hyperparameter...