Machine learning models are increasingly being used within software engineering for their predictions. Research shows that these models’ performance is increasing with new research. This thesis focuses on models for method name prediction, for which the goal is to have a model that can accurately predict method names. With this thesis, we could create a tool that can suggest method names to software developers, which would assist in improving the quality of the projects.
This research aims to get insight into the robustness vulnerabilities of a method name prediction model. We use a genetic search algorithm that looks for these robustness problems. The main question this thesis tries to answer is to what extent the performance metrics are affected by applying metamorphic transformations to the test set of a trained code2vec model. Besides this, this thesis also proposes an alternative metric called percentage MRR, which might better reflect the robustness of a model. The main idea behind this metric is that it penalizes the prediction certainty of a model instead of penalizing the prediction rank.
To answer this research question, a tool is created that runs a genetic algorithm applying these metamorphic transformations to a dataset that a trained model is then evaluating. With this tool, we conducted 22 genetic search experiments on primary metrics and combinations of metrics to see the trade-offs in the Pareto fronts. The guided search of applying metamorphic transformations on the test set results in an average performance decrease of around 19%. This thesis also compares this drop in performance to the performance decrease a random search algorithm would create. Notably, for every transformer added, the average decrease in performance becomes smaller, and there are transformations, e.g., the if-false-else transformation, that have a bigger effect than others. This thesis concludes that the trained model is not robust against metamorphic transformations and has a significant performance drop.

The aim of this project is to improve the scalability of the Opencraft server. The Opencraft server is based on an open-source implementation of the vanilla Minecraft server. This thesis focuses on improving the messaging system of the Opencraft server. The existing implementation of the messaging system is very basic. It went through all online players and promptly generated all messages and send them afterwards, this limits performance and scalability. We decided to replace the system with an implementation of the topic-based variant of the publish/subscribe design pattern. However, we evaluated other options as well. We also decided to implement certain features that were deemed necessary for verification purposes. The server often varied in behaviour compared to the vanilla Minecraft server. This made message verification harder since it was not possible to compare both servers side to side. This lead to the implementation of collision, physics, and water flow, which allows anyone to verify the behaviour of the Opencraft server is correct. The implementation of the messaging system provides a variety of components that can each be configured. All different configurations were tested to find the optimal configuration for the Opencraft server.