Biologically plausible representations have been found to emerge in particular recurrent neural networks when training on path-integration [1, 2]. This report explores factors influencing the occurrence of entorhinal-like representations in recurrent neural networks. Reproducing simplified models from existing studies and created a hybrid model to explore additional factors, including the input features, structural properties, and regularization techniques in recurrent neural networks. Additional experiments evaluate the difference in training performance when entorhinal-like representations are introduced to a recurrent neural network. This report also assesses existing and experimental visualization techniques in their ability to visualize the performance and representation of recurrent neurons. While some experiments show specialized representations, mostly due to regularization; none of the experiments showed typical entorhinal-like representation. These results show how sensitive the emergence of biologically-plausible representations is to network conditions and training procedure,
casting some doubt on the generality of the conclusions proposed in earlier work. ... Read more