Vanishing empirical variance in randomly initialized networks

Neural networks are commonly initialized to keep the theoretical variance of the hidden pre-activations constant, in order to avoid the vanishing and exploding gradient problem. Though this condition is necessary to train very deep networks, numerous analyses showed that it is not sufficient. We explain this fact by analyzing the behavior of the...

How do different explanation presentation strategies of feature and data attribution techniques affect non-expert understanding?: Explaining Deep Learning models for Fact-Checking

The goal of this paper is to examine how different presentation strategies of Explanainable Artificial Intelligence (XAI) explanation methods for textual data affect non-expert understanding in the context of fact-checking. The importance of understand- ing the decision of an Artificial Intelligence (AI) in human-AI interaction and the need for...

Finding Shortcuts to a black-box model using Frequent Sequence Mining: Explaining Deep Learning models for Fact-Checking

Deep-learning (DL) models could greatly advance the automation of fact-checking, yet have not widely been adopted by the public because of their hard-to-explain nature. Although various techniques have been proposed to use local explanations for the behaviour of DL models, little attention has been paid to global explanations. <br/>In response,...

Evaluating Feature Attribution Methods: an Usecase on a Neural Fact-checking Model

In today's society, claims are everywhere, in the online and offline world. Fact-checking models can check these claims and predict if a claim is true or false, but how can these models be checked? Post-hoc XAI feature attribution methods can be used for this. These methods give scores indicating the influence of the individual tokens on the...

Neuro-GOMEA: Using Modern Evolutionary Algorithms to Train Neural Networks

Neural networks (NNs) have, in recent years, become a major part of modern pattern recognition, and both theoretical and applied research evolve at an astounding pace. NNs are usually trained via gradient descent (GD), but research has shown that GD is not always capable of training very small networks. As a result, networks trained via GD are...

Mathematics of Double Descent

Recently, there has been an increase in literature about the Double Descent phenomenon for heavily over-parameterized models. Double Descent refers to the shape of the test risk curve, which can show a second descent in the over-parameterized regime, resulting in the remarkable combination of both low training and low test risk. However, much is...

Detecting climate patterns: Bayesian neural network approach

Machine learning is becoming an increasingly important tool for climate scientists, but hampered by lacking uncertainty quantification. Here, a machine learning approach for detecting patterns indicating a changing climate is combined with probabilistic modelling to retrieve uncertainty values. We train neural networks on climate model...

It sounds like Greek to me: Performance of phonetic representations for language identification

This paper compares the performance of two phonetic notations, IPA and ASJPcode, with the alphabetical notation for word-level language identification. Two machine learning models, a Multilayer Percerptron and a Logistic Regression model, are used to classify words using each of the three notations. With both models the IPA notation outperforms...

Detecting Rhyming Words

Rhyming words are one of the most important features in poems. They add rhythm to a poem, and poets use this literary device to portray emotion and meaning to their readers. Thus, detecting rhyming words will aid in adding emotions and enhancing readability when generating poems. Previous studies have been done on the topic of poem generation....

Population-based Active Learning for Black-Box Regression

Many applications employ models to represent real-life environments efficiently. To allow these models to be realistic it is commonly fitted using a dataset containing labeled samples. When obtaining a label for a sample from the environment is expensive, it is key that the dataset contains only those samples that aid in providing a realistic...

Time-Efficient Video Annotation with t-SNE

Video understanding has received more attention in the past few years due to the availability of several large-scale video datasets and improvement in the computational power of computers. However, annotating large-scale video datasets are cost-intensive due to their complexity. In this work, we propose a time-efficient video annotation method...

Contrastive Learning of Visual Representations from Unlabeled Videos

This thesis presents a novel self-supervised approach of learning visual representations from videos containing human actions. Our approach tackles the complex problem of learning without the need of labeled data by exploring to what extent the ideas successfully used for images can be transferred, adapted and extended to videos for action...

Multimodal information extraction from videos: Automatic creation of highlight clips from political speeches

With the huge amount of data that is collected every day and shared on the internet, many recent studies have focused on methods to make multimedia browsing simple and efficient, investigating techniques for automatic multimedia analysis. This work specifically delves into the case of information extraction from videos, which is still an open...

Learning Scale-Aware Optical Flow

Optical flow is a representation of projected real-world motion of the object between two consecutive images. The optical flow measures the pixel displacement on the image coordinate plane. However, it does not reveal the motion in depth explicitly, which could be useful as input in some tasks such as vehicle tracking. To extend the original...

Learning and Optimizing Probabilistic Models for Planning under Uncertainty

Decision-theoretic planning techniques are increasingly being used to obtain (optimal) plans for domains involving uncertainty, which may be present in the form of the controlling agent's actions, its percepts, or exogenous factors in the domain. These techniques build on detailed probabilistic models of the underlying system, for which Markov...