Online Spatio-Temporal Learning with Target Projection

Conference Paper (2023)
Authors

Thomas Ortner (Zurich Lab)

Lorenzo Pes (Zurich Lab)

Joris Gentinetta (Zurich Lab, ETH Zürich)

C. Frenkel (TU Delft - Electronic Instrumentation)

Angeliki Pantazi (Zurich Lab)

Research Group
Electronic Instrumentation
Copyright
© 2023 Thomas Ortner, Lorenzo Pes, Joris Gentinetta, C. Frenkel, Angeliki Pantazi
More Info
expand_more
Publication Year
2023
Language
English
Copyright
© 2023 Thomas Ortner, Lorenzo Pes, Joris Gentinetta, C. Frenkel, Angeliki Pantazi
Research Group
Electronic Instrumentation
ISBN (electronic)
9798350332674
DOI:
https://doi.org/10.1109/AICAS57966.2023.10168623
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Recurrent neural networks trained with the backpropagation through time (BPTT) algorithm have led to astounding successes in various temporal tasks. However, BPTT introduces severe limitations, such as the requirement to propagate information backwards through time, the weight symmetry requirement, as well as update-locking in space and time. These problems become roadblocks for AI systems where online training capabilities are vital. Recently, researchers have developed biologically-inspired training algorithms, addressing a subset of those problems. In this work, we propose a novel learning algorithm called online spatio-temporal learning with target projection (OSTTP) that resolves all aforementioned issues of BPTT. In particular, OSTTP equips a network with the capability to simultaneously process and learn from new incoming data, alleviating the weight symmetry and update-locking problems. We evaluate OSTTP on two temporal tasks, showcasing competitive performance compared to BPTT. Moreover, we present a proof-of-concept implementation of OSTTP on a memristive neuromorphic hardware system, demonstrating its versatility and applicability to resource-constrained AI devices.

Files

Online_Spatio_Temporal_Learnin... (pdf)
(pdf | 1.36 Mb)
- Embargo expired in 08-01-2024
License info not available