Temporal Fusion Transformer for time series forecasting

The ability to accurately forecast sales volumes holds substantial significance for businesses. Current classical models struggle in capturing the impact of different variables upon the sales volume. These machine learning models are also not applicable to more than one specific product. The Temporal Fusion Transformer (TFT) is implemented to...

What would Jiminy Cricket do?: A pluralist approach in generating and processing morally-aligned text

When making decisions, people are automatically guided by their moral compass. However, AI agents need to be conditioned in order to be steered towards moral behaviour. An environment that can be used to train and test agents is the Jiminy Cricket environment. The Jiminy Cricket environment consists of a set of text-based narrative games, where...

Benchmarking Data and Computational Efficiency of ActionFormer on Temporal Action Localization Tasks: Analysing the Performance and Generalizability of ActionFormer in Resource-constrained Environments

In temporal action localization, given an input video, the goal is to predict which actions it contains, where they begin and where they end. Training and testing current state-of-the-art, deep learning models is done assuming access to large amounts of data and computational power. Gathering such data is however a challenging task and access to...

Evaluating Large Language Model Performance on User and Language Defined Elements in Code

Large Language Models of code have seen significant jumps in performance recently. However, these jumps tend to accompany a notable and perhaps concerning increase in scale and costs. We contribute an evaluation of prediction performance with respect to model size by assessing the layer-wise progression for language and user-defined elements in...

A Cross-Lingual Evaluation of CodeGen's Performance in Code Completion

We present an investigation into the relationship between the average depth of the first correct prediction and the performance of CodeGen. This was done on a dataset comprised of code files comprised of C++, Go, Java, Julia, Kotlin, and Python. The analysis involved investigating the model's predictions at different layers using a Tuned Lens,...

Cross-lingual Performance of CodeGPT on the Code Completion Task

The development of contemporary source code auto-completion tools have significantly boosted productivity and efficiency of developers. In 2021, the GPT-2-based Transformer CodeGPT was developed to support code completion and text-to-code generation. Similarly to most code models however, CodeGPT was trained on a limited set of widely-used...

Compressing code generation language models on CPUs: Using Group Lasso pruning and post-training quantization

Code generation models have become more popular recently, due to the fact that they assist developers in writing code in a more productive manner. While these large models deliver impressive performance, they require significant computational resources and memory, making them difficult to deploy and expensive to train. Additionally, their large...

Does text matter?: Extending CLIP with OCR and NLP for image classification and retrieval

Contrastive Language-Image Pretraining (CLIP) has gained vast interest due to its impressive performance on a variety of computer vision tasks: image classification, image retrieval, action recognition, feature extraction, and more. The model learns to associate images with their descriptions, a powerful method which allows it to perform well on...

Error Estimates for Finite Element Simulations Using Neural Networks

In this work residual error estimates are constructed using Neural Networks for Finite Element Method. These can be used to do adaptive mesh refinement. Two neural networks are developed the Multilayer Perceptron and the Transformer model. The error estimates are made for 1d poisson equations but the idea will generalise to higher dimensions as...

DilBERT^2: Humor Detection and Sentiment Analysis of Comic Texts Using Fine-Tuned BERT Models

The field of Natural Language Processing (NLP) techniques has progressed rapidly over the recent years. With new advancements in transfer learning and the creation of open-source projects like BERT, solutions and research projects emerged implementing new ideas in a variety of domains, for tasks including text classification or question...