To be able to understand the dynamic driving environment, an autonomous vehicle needs to predict the mo- tion of other traffic participants in the driving scene. Motion prediction can be done based on experience and recently observed series of past events, and entails reasoning about probable outcomes with these past ob- servations. Aspects that influence driving behavior comprise many factors, such as general driving physics, infrastructure geometry, traffic rules, weather and so on. Models that are used today are incapable of including many of these aspects. These deep learning models often rely heavily on the past driven trajectory as an information source for future prediction. Sparsely, some work has been done to include interaction between vehicles or some infrastructural features to the model. The lack of information regarding the driving scene can be seen as a missed opportunity, because it is a use- ful feature source to predict the motion of a vehicle. Especially when a map of the driving scene is available, reliable information regarding the road layout can be extracted. In this thesis, a model architecture is sought that is aware of the interaction between vehicles, and that un- derstands the geometry of the roads in the scene. Importantly, map information regarding the driving scene should be used as the primary source of information regarding the road geometry. First, a baseline deep learning model is constructed, that can generate predictions based on the past observed trajectory. To add interaction features to the model, a social pooling module is introduced. The social pooling method allows to efficiently include the driving behavior of other vehicles in the scene. To introduce reliable, map-based road information to the model, two novel methods are proposed. In the first, the predictions from a model are used to extract features in a map. These features describe the road scene around the predicted location, and are used to update these predictions. In the second method, the semantic map is only used to extract a road segment ahead of the vehicle. A road-RNN is introduced to con- struct features regarding the road segment, and an attention mechanism to determine what part of the road segment is relevant for the predictions. These modules are referred to as road-refinement and road-attention respectively. The importance of including both road geometry and interaction methods in the model is shown by con- structing 5 different models that vary in their road-geometric and interaction awareness. A baseline deep learning model is used and extended with a road-geometry module, an interaction module, a combination of the two, or none. To test the prediction capabilities of the models, they are trained on two different datasets. The first dataset, called i80, consists of trajectory recordings from a straight highway with dense traffic. The other dataset is a curved version of the i80, called i80c, where the trajectories and road are transformed to introduce road-geometric variations in the data. The prediction performances from the models clearly show the importance of both interaction-aware and road-geometry aware modeling. The road-refinement and road-attention models outperform the road-agnostic baseline model on the i80c data, showing better understanding of the road layout. Comparison between these models learns that the road-attention model is more effective for road-geometry inclusion. Additionally, the performance increase for interaction-aware models compared to the baseline clearly shows the importance of interaction in modeling on both datasets.The model that combines the interaction and road-attention modules shows outstanding prediction performance on the challenging i80c dataset compared to all other models. From the predictions it can be seen that the model understands the road layout ahead of the subject vehicle, as well as the interactive forces that are in play.