Print Email Facebook Twitter Sequential Monte Carlo method for training Neural Networks on non-stationary time series Title Sequential Monte Carlo method for training Neural Networks on non-stationary time series Author Hoogendoorn, Jasper (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Oosterlee, Kees (mentor) Borovykh, Anastasia (graduation committee) Degree granting institution Delft University of Technology Programme Applied Mathematics | Financial Engineering Date 2019-06-28 Abstract In this thesis, we study the sequential Monte Carlo method for training neural networks in the context of time series forecasting. Sequential Monte Carlo can be particularly useful in problems in which the data is sequential, noisy and non-stationary. We compare this algorithm against a gradient-based method known as stochastic gradient descent (SGD), a commonly used method for training neural networks. The performance of SGD on forecasting non-stationary, noisy time series can be poor due to the possibility of overfitting on the data. The sequential Monte Carlo method may offer a solution for the problems that arise in forecasting non-stationary time series with SGD neural networks. At the same time, neural networks trained with SGD give deterministic predictions, and there is a need for quantification of the uncertainty in the prediction. Sequential Monte Carlo sequentially samples the weights of the neural network, providing a posterior distribution on the weights and thus the outcome. In this work, the sequential Monte Carlo algorithm is tested and analyzed, with different parameter settings, on four time series to give an overview of the behavior. Furthermore, we apply the SMC algorithm on a convolutional neural network known as WaveNet. We show that the SMC algorithm is very well-suited for forecasting non-stationary time series, and can significantly outperform the gradient-based SGD method. Additionally, we show that for specific time series the SMC algorithm on a convolutional neural network outperforms the SMC algorithm on a fully-connected neural network. Subject sequential Monte CarloNeural NetworksTime Series ForecastingConvolutional Neural Network To reference this document use: http://resolver.tudelft.nl/uuid:659e9fd5-d251-46fe-8455-3a17bdd4f48c Part of collection Student theses Document type master thesis Rights © 2019 Jasper Hoogendoorn Files PDF Thesis_Jasper_Hoogendoor_FINAL.pdf 1.87 MB Close viewer /islandora/object/uuid:659e9fd5-d251-46fe-8455-3a17bdd4f48c/datastream/OBJ/view