Modeling the relationship between rainfall and runoff is a longstanding challenge in hydrology and is crucial for informed water management decisions. Recently, Deep Learning models, particularly Long short-term memory (LSTM), have shown promising results in simulating this relationship. The Transformer, a newly proposed deep learning architecture, has also demonstrated the ability to outperform LSTM in machine translation, text classification, etc. However, there has been limited research on applying Transformers for rainfall-runoff modeling.

The research examined the performance of using Transformer architecture, including its time series forecasting variants, to develop rainfall-runoff models using the CAMELS (US) data set. These models were compared to the LSTM regional rainfall-runoff models, with a particular focus on snow-driven basins as the attention mechanism in Transformer is believed to allow it to attend to the earlier precipitation events in the meteorological forcing. Additionally, the Transformer's potential as a global rainfall-runoff model was also tested using the global Caravan data to determine if it could learn and generalize a wide range of rainfall-runoff behaviors, allowing it to potentially be applied in ungauged basins.

The results suggest that while Transformer and its variants may not be able to fully replace LSTM for rainfall-runoff modeling, the variant called Reformer has shown promise for daily discharge forecasting in snow-driven basins, particularly in terms of peak flow and low flow prediction. However, using the global Caravan data for building a global rainfall-runoff model was not successful due to uncertainty in the forcing data, particularly precipitation. The code for Transformer-based rainfall-runoff modeling is available publicly at https://github.com/Numpy-Panda/neuralhydrology_Transformer.

Finite Element Analysis is an application used in many engineering fields, including geotechnical engineering. With increasing complexities of structures and material behavior, classical approach for non-linear finite element analysis has become computationally expensive (Gulikers, 2018). In order to improve the efficiency and accuracy of the model, Machine Learning (ML) is introduced. The ever-increasing accessibility and magnitude of computational power has made machine learning gained popularity since 1990s. Artificial Neural Networks have especially been proven to be a versatile machine learning framework, capable of solving complex tasks such as voice recognition to solving failure analysis of structures. This thesis aims at implementing soil constitutive models such as Linear Elastic and Elasto-plastic models and integrating them into a Finite Element Engine. Two different approaches have been attempted to implement the ANN into the constitutive models. The first approach involves the implementation of a "Generic" Neural network to simulate the natural anisotropic behavior of soils. Datasets have been generated with the help of theoretical soil tests classified as ‘Standard’ and ‘Non-standard’ tests. The standard tests depict an isotropic behavior, where the strains and stresses in different directions are proportionate. Non-standard tests consist of data representing anisotropic behavior where the strain increments in every direction are randomized. The neural network has first been implemented for a linear elastic soil model. Itis observed that when the network is trained and tested with similar kind of data, predictions are accurate. However, when the neural network is trained with a certain type of data (such as standard test), it is unable to predict the response for the other data (non-standard test). When the neural network (trained with standard data) is subsequently implemented in Finite Element analysis by deriving and substituting the material stiffness matrix from the neural network in a constitutive model, the difference between the isotropic linear elastic matrix (for linear elastic model) and the matrix derived from the neural network lead to significant errors, despite the accurate predictions of the neural network. The second approach is a component based approach in which the neural network is modeled to learn the typical behaviors of soil such as linear elasticity and plasticity. The data for training and testing the neural network are generated using standardized laboratory tests (CRS, DSS, Biaxial etc.). The neural network is modeled for linear elastic soil, by applying certain weight constraints. It is observed that the neural network has accurate predictions. Further implementation in constitutive model shows that the neural network is able to mimic the behavior of Linear elastic soil model. The neural network model, is subsequently implemented to predict Mohr-Coulomb behavior, and the predictions are significantly accurate. Further the generalization ability of the network is analyzed to evaluate the practical applicability of the network, followed by a sensitivity analysis to get an understanding about the influencing parameters in the network. The devised neural network, when implemented in a constitutive model displays a behavior similar to the theoretical constitutive model of Mohr-Coulomb soil. Finally, this thesis concludes with the implementation of the neural network constitutive model in Finite Element Analysis where it leads to convergence errors in the model.

Rainfall-runoff modelling is essential for short- and long-term decision-making in the water management sector. The accuracy of streamflow predictions of hydrologic models increases with the availability of and the access to streamflow observations. Therefore, one of the key challenges in the field of hydrology is to produce Predictions in Ungauged Basins (PUB), where observations are lacking. Recent research has shown the potential of deep learning neural networks as an alternative approach to conceptual and process-based hydrologic models for this purpose. 
In this study, the existing Multi-Timescale LSTM (MTS-LSTM) architecture is used to investigate if such a deep learning network is able to learn universal hydrologic behaviour. Therefore, a MTS-LSTM is trained on a large variety of >500 US catchments and subsequently tested outside the US, in the European Meuse river basin. The model is not retrained or finetuned to simulate an ungauged situation and to asses whether the streamflow predictions can compete with those from the uncalibrated distributed model wflow_sbm. 
Results indicate that the MTS-LSTM trained on US data cannot compete with wflow_sbm in the Meuse catchments. The simulated streamflow time series can be unrealistically shifted and scaled compared to the time series of observed streamflow due to sensitivity regarding static model input. This means, the values for catchment characteristics cannot be extremer in the testing data than in the training data. Therefore, it is recommended to select catchments for the model training such that the most extreme conditions are covered. In the case that the MTS-LSTM is trained for a specific region, results clearly compete with or outperform the distributed model. For the Meuse test catchments, the neural network achieves Nash-Sutcliffe Efficiency (NSE) values>0.46 where the application of wflow_sbm is problematic and yields negative NSE values. To exploit the potential of an LSTM, the model should be trained on all available data of the entire Meuse basin instead of on the subset of catchments used here. For some water management applications it is important to accurately predict high flow events. Training the MTS-LSTM with the Mean Quadrupled Error (M4SE) loss function showed that the peak flow representation can improve for catchments where the use of a NSE loss already leads to good predicting performance. Thus – for and ungauged catchments – an implementation of a combined loss function appears a valuable follow-up research. 
For ungauged catchments, these results imply that the global neural network model as tested in this study should be supplemented with finetuning. Thereby, the global model could not yet be applied everywhere, however, in regions where only few years of streamflow records are available. Alternatively, a regional neural network model trained on nearby catchments could be applied for PUB, if streamflow observations are accessible in the surrounding area. Finally, it is of high importance to maintain and extend the network of streamflow gauging stations globally, and to ensure easy access to the data.