Large Language Models of code have seen significant jumps in performance recently. However, these jumps tend to accompany a notable and perhaps concerning increase in scale and costs. We contribute an evaluation of prediction performance with respect to model size by assessing the layer-wise progression for language and user-defined elements in code, using a new technique of Tuned Lenses. We show that language-defined elements can be predicted more accurately in earlier layers of the PolyCoder model than user-defined elements and contribute an evaluation of the attention mechanism, which shows patterns that explain such aspects of performance and indicate areas of missed potential. These findings encourage research into the internal prediction performance for other characteristic aspects of code and could lead to the introduction of new methods that make use of these characteristics to improve performance without relying on scaling.

We present an investigation into the relationship between the average depth of the first correct prediction and the performance of CodeGen. This was done on a dataset comprised of code files comprised of C++, Go, Java, Julia, Kotlin, and Python. The analysis involved investigating the model's predictions at different layers using a Tuned Lens, which enables examining the intermediate representations. Additionally, attention heads were examined to gain insights into the model's behavior. We found that there is a subset of four layers in which tokens are predicted correctly for the first time. These peaks are evident in CodeGen's performance and come after a small dip, a dip that is present in the last layer. The results shed light on the varying performance of different layers and provide valuable insights into the strengths and weaknesses of CodeGen. These findings contribute to our greater understanding of language model performance in code completion tasks and provide implications for future improvements in this domain.

The development of contemporary source code auto-completion tools have significantly boosted productivity and efficiency of developers. In 2021, the GPT-2-based Transformer CodeGPT was developed to support code completion and text-to-code generation. Similarly to most code models however, CodeGPT was trained on a limited set of widely-used languages (Java, Python) - leading to constrained efficacy in lower-resource languages. This motivated us to research CodeGPT's performance on the token-level code completion task across high- and low-resource languages. We investigate in which scenarios CodeGPT predicts incorrect tokens with high certainty using a tuned lens, followed by studying attention patterns that underlie the observed behaviour. Our findings indicate that CodeGPT is most competent in Java and Python code (Top-1 accuracies: 69.2% and 68.2% respectively). It generates false predictions with highest confidence when it encounters unfamiliar constructs in low-resource languages, or code structures that cannot be predicted from left context only. Moreover, we find a positive correlation between null attention and model confidence.

In recent years, deep learning techniques, particularly transformer models, have demonstrated remarkable advancements in the accuracy and efficiency of language models. These models provide the foundation for many natural language processing tasks, including code completion. The effectiveness of code completion models has been the subject of a variety of empirical studies. However, none of the existing literature has explicitly investigated the potential impact of common code structures on the performance of large language models during code completion. This paper evaluates the influence of common code structures on the code completion performance of CodeParrot, a state-of-the-art natural language processing model. Using the tuned lens method, we show that typical code structures lead to a higher completion accuracy compared to uncommon code structures, due to their frequent occurrence, consistent syntax, clear semantics, and contextual clues. Finally, we perform an attention investigation to assess the significance of the common code structures and reveal potential data patterns across low- and high-resource languages.

Artificial intelligence systems assist humans in more and more cases. However, such systems' lack of explainability could lead to a bad performance of the teamwork, as humans might not cooperate with or trust systems with black-box algorithms opaque to them. This research attempts to improve the explainability of artificial intelligence systems by proposing a framework which models human workload in a value and tailors explanations to this value. Such explanations could provide agents' confidence, causes of making decisions and counterfactual parts to support their suggestions and are adjusted according to agents' knowledge of humans. Results show that adjusted explanations could improve participants' subjective trust in agents and make participants' take more suggestions, while no impact on collaboration fluency or teamwork performance is found.

Nowadays, artificial intelligence (AI) systems are embedded in many aspects of our lives more than ever before. Autonomous AI systems (agents) are aiding people in mundane daily tasks, even outperforming humans in several cases. However, agents still depend on humans in unexpected circumstances. Thus, the main goal of these agents has transformed from becoming independent to interdependent systems, collaborating with humans. This collaboration is far from perfect and could be improved in several aspects. Communication is crucial for flawless collaboration and its key aspect is explainability. This paper studies the impact of tailoring explanations according to human performance in a well-defined collaborative human-agent teaming (HAT) urban search-and-rescue (USAR) task environment. A controlled experiment was conducted in a between-subject manner, with two different agent implementations, where it was hypothesised that when an agent provides explanations tailored to human performance, the collaborative performance, the trust towards the agent and the individual satisfaction of the human would increase. Results of the experiment confirmed that this is indeed the case for explanation satisfaction, however, not necessarily for trust and performance metrics. The conclusions also included that the tailoring resulted in a decreased collaborative performance. The research contributes to the bigger picture of how tailoring explanations to various factors, would have an impact on the overall collaborative performance and systematic actualisation of HAT.

Aligning human trust to correspond with an agent's trustworthiness is an essential collaborative element within Human-Agent Teaming (HAT). Misalignment of trust could cause sub-optimal usage of the agent. Trust can be influenced by providing explanations which clarify the agent's actions. However, research often approaches explanations statically, making them not adjustable to real-time situations. In this research, we study the effectiveness of an agent capable of modelling human trust and tailoring explanations to influence it. We achieve this by modifying an existing HAT environment and setting up an experiment comparing a trust and baseline agent. Modelling human trust is calculated through the number of suggestions ignored. When the model estimates low trust, more explanation types are used during communication. Higher trust uses fewer explanation types in order to save time. However, the results indicate no difference between the baseline and trust agent rejecting the hypothesis. A potential cause for the rejection can be found in either a flaw in the agents' design or information overload.

This paper tries to combat the food waste of strawberries during the harvesting steps.
An automatic pipeline must be established to combat this food waste.
One of the steps needed in this pipeline is detecting strawberries in images.
Therefore, this paper aims to find out which Convolutional Neural Network (CNN) can be best used to detect strawberries.
Faster r-cnn, Mask r-cnn and RetinaNet are compared against each other using different setting.
Mask r-cnn achieved the highest average bounding box and segmentation mAP with 51.63 and 73.20 respectively.

To reduce food waste, the strawberry harvesting process should be optimized. In the modern era, computer vision can provide huge amounts of help. This paper focuses on optimizing pre-trained convolutional neural networks (CNN) to determine the maturity level of strawberries on a 1-10 scale. Here, 1 means unripe and 10 means overripe. Maturity level 8 is marketable. Experiments are done with VGG19, Resnet50, InceptionV2, Alexnet, and EfficientNetB2 as classifiers on segments using ADAM and SGD as optimizers and cross-entropy as loss function. The same CNN's are applied as a backbone for FasterRCNN to see how they would behave within an object detection architecture. The biggest challenge during this research was the low amount of training data. The research showed that using convolutional neural networks as a maturity level predictor is possible, but a well made training set with an equal spread for each maturity level is necessary to possibly achieve high accuracy.

The purpose of this research is to reduce food waste by monitoring the ripening process of strawberries in order to optimize the harvesting time. To improve the moment of harvest, we need to know the ripeness of a strawberry. Using data from different color ranges and spaces we should be able to predict the ripeness of a strawberry on a 1-10 scale. We want to answer the question whether data from multiple color spaces can improve such a prediction model.

The prediction is performed on strawberry segments, using linear regression. The regression is performed based on the ripeness and the red and green pixel values in a segment. Each color space uses a slightly different metric.

We are able to show that the YCbCr and CIELab color space outperform RGB in such a linear regression. This is likely to come from the fact that these color spaces separate the luminance and chrominance. For the near-infrared range however, we do not have enough data to make such a conclusion as the available data only has ripeness levels 7-9.

To reduce food waste, it is important to know what strawberries to prioritise for harvesting. Size is an important quality attribute for strawberry. In order to know the size, the depth of the strawberry in the image must been known. To estimate the depth, stereovision gets utilized using binocular images. Since classic stereovision methods are quite inaccurate in predicting small areas in images, a technique by [Mustafah et al., 2013] is used. Using given segments of the strawberries, the left and right strawberry images will get matched. With the matched strawberries, the disparity can be calculated and thus the depth. Using the depth, the size can be estimated.

Ever since the invention of the Internet, more and more computers are connected throughout the world. Though this has brought numerous new inventions used every day, like social media, e-commerce, and video conferencing, it also opens up new opportunities for cyber criminals. As the intrusion detection systems used to identify malicious behavior in a computer network can generate large amounts of alerts, methods have been developed to aid security analysts in gaining insights into what is happening on the network. Of course, there is always room to improve these methods, which is the topic of this thesis. Currently, one of the state-of-the-art methods uses state machines to model the alert sequences. State machines are a good fit as they can extract the context of different alerts, but they cannot extract information like parallelism between different alerts. That is where field process mining comes in, with process mining algorithms being able to extract parallelism from sequential data. In this thesis, state-of-the-art algorithms from process mining are evaluated for modeling alert datasets from intrusion detection systems with the aim of improving the current methods. As a comparison, different methods for learning state machines also tested for the same data. The results of the evaluation and comparison show that the state machines perform better in modeling the alert datasets with respect to explaining the data. On the other hand, thee process mining algorithms were not able to construct sound models for the datasets, and a fourth mining algorithm gave false implications about the data. Furthermore, the possibility of combining state machines with process mining was also tested, with the idea that the combination can use the state machines to extract context and the process miner to extract parallelism. This method did not yield any improvements for the alert datasets tested, but that does not mean it is not viable in other cases.

MalPaCa is an unsupervised clustering tool, which the main purpose is to cluster unidirectional network connections based on network behavior. The clustering is only based on non-intrusive (private) packet features such as transport and network header fields, and thus it has a strong potential use-case. This paper focuses on feature extraction and finding the best combinations that provide best clustering results. The features should be generalizable to a wide range of malware families and follow an easy extraction process. To expand the research one additional packet-based feature is found, TCP flags,  as well different variants of previously extracted features were employed, which improves the efficacy of the tool. Finally, a grid search is performed to determine the best combination of the features.