On the Emergence of Biologically-Plausible Representation in Recurrent Neural Networks for Navigation

On the Emergence of Biologically-Plausible Representation in Recurrent Neural Networks for Navigation

Zeeuwe, Daan (TU Delft Electrical Engineering, Mathematics and Computer Science)

Broekens, Joost (mentor)

Delft University of Technology

Computer Science

2019-09-30

Biologically plausible representations have been found to emerge in particular recurrent neural networks when training on path-integration [1, 2]. This report explores factors influencing the occurrence of entorhinal-like representations in recurrent neural networks. Reproducing simplified models from existing studies and created a hybrid model to explore additional factors, including the input features, structural properties, and regularization techniques in recurrent neural networks. Additional experiments evaluate the difference in training performance when entorhinal-like representations are introduced to a recurrent neural network. This report also assesses existing and experimental visualization techniques in their ability to visualize the performance and representation of recurrent neurons. While some experiments show specialized representations, mostly due to regularization; none of the experiments showed typical entorhinal-like representation. These results show how sensitive the emergence of biologically-plausible representations is to network conditions and training procedure,
casting some doubt on the generality of the conclusions proposed in earlier work.

Recurrent Neural Networks
Regularization
exploratory research
Emerging behavior
Entorhinal-like representation
artificial navigation
neural network analysis

http://resolver.tudelft.nl/uuid:72a82d88-3e70-4bbc-91ec-59a4f946ca02

Student theses

master thesis

© 2019 Daan Zeeuwe