Activating frequencies

Exploring non-linearities in the Fourier domain

Master thesis (2018)

Authors

W.R. Uijens Electrical Engineering, Mathematics and Computer Science

Contributors

J.C. van Gemert (supervisor 1)
M.J.T. Reinders (supervisor 2)
K.A. Hildebrandt (supervisor 2)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2018 Wouter Uijens
Machine Learning Activation Function Fourier Transform
More Info
expand_more

Published Date

13-04-2018

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Convolutional Neural Networks (CNNs) are achieving state of the art performance in computer vision. One downside of CNNs is their computational complexity. One way to make CNNs more computational efficient is by implementing their convolutions in the frequency domain, using Fast Fourier Transforms (FFTs). This has as a consequence that most computational time in modern CNNs is spent in those FFTs.
If all components of the CNN algorithm could be implemented in the frequency domain, it would no longer be necessary to go back at all to the spatial domain. Most components of the CNN do have alternatives that can be implemented in the frequency domain, however one crucial component doesn’t: the activation function. This thesis is the first published study into activation functions in the frequency domain.
In this thesis several potential candidates for an activation function that works directly in the frequency domain are studied. Furthermore, some theoretical contributions on this subject are made.