In this thesis, we study the sequential Monte Carlo method for training neural networks in the context of time series forecasting. Sequential Monte Carlo can be particularly useful in problems in which the data is sequential, noisy and non-stationary. We compare this algorithm against a gradient-based method known as stochastic gradient descent (SGD), a commonly used method for training neural networks. The performance of SGD on forecasting non-stationary, noisy time series can be poor due to the possibility of overfitting on the data. The sequential Monte Carlo method may offer a solution for the problems that arise in forecasting non-stationary time series with SGD neural networks. At the same time, neural networks trained with SGD give deterministic predictions, and there is a need for quantification of the uncertainty in the prediction. Sequential Monte Carlo sequentially samples the weights of the neural network, providing a posterior distribution on the weights and thus the outcome. In this work, the sequential Monte Carlo algorithm is tested and analyzed, with different parameter settings, on four time series to give an overview of the behavior. Furthermore, we apply the SMC algorithm on a convolutional neural network known as WaveNet. We show that the SMC algorithm is very well-suited for forecasting non-stationary time series, and can significantly outperform the gradient-based SGD method. Additionally, we show that for specific time series the SMC algorithm on a convolutional neural network outperforms the SMC algorithm on a fully-connected neural network.

Clients with a mortgage loan may prepay a part of their loan before the contractual date. This is called prepayment. In the case of a prepayment, the bank who issued the loan earns less interest than ini- tially agreed. It is therefore essential to build accurate models for predicting prepayment behavior. In this thesis, machine learning models called artificial neural networks are used to predict conditional prepayment rates. In particular, the possibility to quantify the model uncertainty is studied. It is important to acknowledge that the accuracy of a model is not constant over its domain. Most ma- chine learning methods do not provide information about the uncertainty regarding a prediction and, unfortunately, methods that do so are often computationally expensive. This thesis studies uncertainty estimates generated by a neural network with dropout applied. Dropout is a method that randomly drops out neurons in each layer and has been suggested to prevent overfitting. We will see that this method can also be used to extract a measure of uncertainty regarding the prediction of a neural net- work efficiently. This method is used first to evaluate uncertainty for three simple functions under different distributions of the train data. Then the uncertainty estimates are evaluated for a neural network that predicts conditional prepayment rates. It will be illustrated that the uncertainty estimates provide an accurate indication of regions of the domain where predictions are inaccurate. Moreover, using a different model in the regions identified as uncertain can result in higher model performance.

Since the liberalization of the energy markets, the storage of energy is decoupled from the production and sales. In Western-Europe the storage of natural gas becomes more and more important because production fields get depleted and governments force companies to slow down their production because of tremors in the ground. Natural gas needs to be imported from countries that are far away, like for example Russia. To provide in security of supply and to ensure there is enough natural gas when the demand is high, it is important to store natural gas nearby.

To determine the value of a gas storage facility in a reliable way we need an efficient market. For an efficient market is needed that the financial instruments, like futures contracts and options on natural gas, are liquidly traded on the exchange. If this condition is met, we are able to determine the value of storage according to market prices.

The COS method was already presented as an efficient method for pricing a broad spectrum of financial derivatives and can be used in combination with all processes for the underlying for which a characteristic function is known. For processes whose characteristic function is not available, the adjoint expansion method can be used to obtain an approximation of the characteristic function. In this work the COS method will be presented as an efficient method for determining the value of gas storage contracts which is competitive with existing valuation methods for natural gas storage contracts.

Recent works have shown that neural networks can be employed to solve partial differential equations, bringing rise to the framework of physics informed neural networks.The aim of this project is to gain a deeper understanding of these novel methods, and to use these insights to further improve them. We show that solving a partial differential equation can be formulated as a multi-objective optimization problem, and use this formulation to propose several modifications to existing methods. These modifications manifest as a scaling parameter, which can improve the accuracy by orders of magnitude for certain problems when it is chosen properly. We also propose heuristic methods to approximate the optimal scaling parameter, which can be used to eliminate the need to optimize this parameter. Our proposed methods are tested on a variety of partial differential equations and compared to existing methods. These partial differential equations include the Laplace equation, which we solve in up to four dimensions, the convection-diffucsion equation and the Helmholtz equation, all of which show that our proposed modifications lead to enhanced accuracy.