Convolutional Neural Networks (CNNs) are achieving state of the art performance in computer vision. One downside of CNNs is their computational complexity. One way to make CNNs more computational efficient is by implementing their convolutions in the frequency domain, using Fast
...
Convolutional Neural Networks (CNNs) are achieving state of the art performance in computer vision. One downside of CNNs is their computational complexity. One way to make CNNs more computational efficient is by implementing their convolutions in the frequency domain, using Fast Fourier Transforms (FFTs). This has as a consequence that most computational time in modern CNNs is spent in those FFTs.
If all components of the CNN algorithm could be implemented in the frequency domain, it would no longer be necessary to go back at all to the spatial domain. Most components of the CNN do have alternatives that can be implemented in the frequency domain, however one crucial component doesn’t: the activation function. This thesis is the first published study into activation functions in the frequency domain.
In this thesis several potential candidates for an activation function that works directly in the frequency domain are studied. Furthermore, some theoretical contributions on this subject are made.
