Meta-learning for low-cost adaptive neuromorphic computing with memristors

Master Thesis (2024)
Authors

K. Pinto (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Supervisors

C. Frenkel (TU Delft - Electronic Instrumentation)

Faculty
Electrical Engineering, Mathematics and Computer Science
More Info
expand_more
Publication Year
2024
Language
English
Graduation Date
26-08-2024
Awarding Institution
Delft University of Technology
Programme
Electrical Engineering
Faculty
Electrical Engineering, Mathematics and Computer Science
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Traditional computing approaches based on the von Neumann architecture consist of physically separate storage and computation units. This requires the data to be moved back and forth between the storage and computation units, resulting in increased latency and energy costs known as the memory-wall bottleneck. To address this issue, in-memory computing has emerged as a possible solution wherein computation is performed directly inside the memory units, similar to the working of the human brain. Memristor devices arranged in a cross-bar array formation are promising candidates for such in-memory accelerators as they enable direct processing of data within memory. Despite the promises shown by memristors, they still suffer from non-idealities which hinder their application as neural network accelerators such as asymmetric conductance update, device-to-device variations, and temporal drift. While it is possible to optimize the parameters of the neural network in the cloud to reduce the impact of these non-idealities, the drawback of this approach is that the trained model is unable to adapt to new tasks. This thesis proposes using a bi-level optimization scheme such as meta-learning to overcome the effect of memristor non-idealities. Meta-learning (or learning-to-learn) is a field of machine learning that allows neural networks to learn from multiple tasks and use that experience to improve performance on future tasks. Using the model agnostic meta-learning (MAML) algorithm, it is possible to train neural networks such that the impact of memristor non-idealities is reduced. Incorporating memristor dynamics in a traditional machine learning setup resulted in a classification accuracy of up to 59.20% on the Omniglot dataset with a 5-layer convolutional neural network, a sharp decrease from the baseline accuracy of 97.5%. This thesis demonstrates that using MAML on the Omniglot dataset for the five-way one-shot learning task results in neural networks achieving a classification accuracy of up to 97.5% when considering asymmetric conductance updates of memristors, which is equivalent to the accuracy obtained when training the neural network using ideal memristor dynamics. Furthermore, when taking into account device-to-device variations, accuracy values of up to 87.5% are obtained. Finally, when accounting for temporal drift, accuracy values greater than 70% were maintained for up to 11 inference steps using MAML when considering the ratio of learning steps to the temporal drift to be 1:100.

Files

License info not available
warning

File under embargo until 26-08-2026