The ability to model other agents can be of great value in multi-agent sequential decision making problems and has become more accessible due to the introduction of deep learning into reinforcement learning. In this study, the aim is to investigate the usefulness of modelling other agents using variational autoencoder based models in partially observable settings. Previous studies that model other agents using (variational) autoencoders have shown promising results. In these studies, a single protagonist agent learns representations of other agents to then use them as additional components of its observation space which is, as such, augmented with those representations. It is, however, not always entirely clear what is being modelled and what would be the best feature of the other agent to represent. Moreover, in these works, a comparison between the used variational autoencoder based models and a baseline classifier trained to solve the same classification task is missing. This study investigates which features can best be used for the augmentation of the observations of deep reinforcement learning agents and if these features can be represented by variational autoencoder based models. Subsequently, it compares these models with a baseline classifier that solves the same classification problem to find out which model yields the best results when used for augmenting observations. Overall, the results suggest that it is beneficial to augment the observations of deep reinforcement learning agents with features related to other agents learned in a pre-training phase. Another interesting result is that the baseline classifier achieves similar or better performance compared to the variational autoencoder based model. Further research needs to be conducted to confirm the soundness of these findings.

Motivation: Traffic forecasting is becoming a vital component of our travel experience. It plays a key role in intelligent transportation systems that allow us to make smarter use of existing transportation networks. This study focuses on the possible role of artificial neural networks in these systems and what data can be best feed in to them to retrieve the best results. Aim: The goal of this study is to see whether two layered feed forward neural networks outperform the statistical ARIMA model in motorway traffic forecasting. In specific, whether or not the usage of upstream and multivariate data decreases the forecasting errors of the neural network, how this relates to the amount of samples used as input, and how this relates to the amount of time steps that is forecasted ahead. Results and conclusions: Two different traffic networks are used to train and test the models. The testing results show that, when doing predictions using time steps covering 10 minutes of traffic data and forecasting one time step ahead, the optimal amount of samples used as input is 4. Increasing the input length after this does not result in better predictions, it even slightly increases the prediction errors. Moreover, it became clear that up to 3 or 4 time steps forecasting in the future, the neural networks using upstream data outperform the ARIMA model. After this an ARIMA model that uses deseasonalized data or a neural network that uses deseasonalized data is a better option. There is always a two layered neural network that outperforms the ARIMA models. Furthermore, the usage of upstream data almost always decreases the prediction errors. This is different with the usage of multivariate data, which hardly contributes to a better prediction in the used form.