The introduction of large language models (LLMs) has transformed the way software is written. With the help of LLM powered code generation the productivity of software engineers has increased all over the world. However, these models are also computationally expensive. The ubiquitous use of these models has raised significant sustainability concerns.

LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.

In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial. ... Read more

The adoption of AI systems across various sectors has increased considerably in recent years. This is a consequence of the remarkable capability of AI to extract insights from large-scale datasets, improve personalization, automate tasks and complex processes within organizations, and support more informed decision-making. Notable examples include the financial sector, where AI is applied to monitor transactions and accelerate credit decision processes; healthcare, where AI contributes to drug discovery and assists clinicians in the early diagnosing; manufacturing, where predictive maintenance using AI systems help reduce costs and mitigate the risks associated with unexpected failures; and software engineering, where AI supports anomaly detection, fault prediction, and resource demand forecasting in large-scale, complex systems.
Despite the widespread adoption and potential of AI systems, most research has been focused on model development, while investigations into their lifecycle and evolution in production environments remain at an early stage. This research path is particularly relevant for AI practitioners, who are responsible for ensuring the reliability, functionality, and predictive accuracy of deployed systems. To bridge the gap between scientific research and the practical needs of industry practitioners, this thesis focuses on two key aspects of the AI lifecycle: techniques for monitoring and maintaining AI systems over time.....
... Read more

This thesis investigates reducing carbon emissions in code generation using large language models (LLMs) by comparing function-level and line-level code completions across models of different sizes (1.5B and 9B parameters). The study utilises the BigCodeBench dataset, comprising 1,140 Python programming problems, to evaluate the energy consumption, test accuracy, and time efficiency of code completions. The models, 4-bit quantised and run on a CPU, performed 30 function-level completions and 30 line-level completions for each line, which were tested for correctness. Results indicate that, while line-level completions require slightly more energy per token, they are more efficient overall in terms of total energy consumption and token usage. The smaller model with line-level completions showed significant reductions in carbon emissions, achieving an average tenfold reduction compared to the large model with function-level completions. With the large model, line-level completions achieved a $4.5\times$ reduction in carbon emissions compared to function-level completions. Line-level completions were more token-efficient, wasting less than 1\% of energy, compared to 20\% for function-level completions. From a sustainability perspective, line-level completions offer a practical strategy to reduce the environmental impact of code generation tasks while maintaining strong performance. The study suggests that optimising completion strategies could help balance energy consumption, test accuracy, and time efficiency. Future research could explore a broader range of model sizes, fine-tuning models specifically for line-level completions, a performance decrease in solution length, and alternative validation metrics to assess code generation performance. ... Read more

Understanding the energy profile of a complex, multi-faceted software system is difficult. In this thesis, we present a novel methodology, called SMURF, a five-step methodology that gives insights into the energy consumption of a complex system. The methodology is broadly applicable, supports informed decision-making, and closely involves and engages stakeholders. We evaluate the methodology with a case study on MUST, a software system used in spacecraft operations. In the case study, SMURF successfully finds energy hotspots and wasteful components in MUST, and is used effectively to formulate actionable recommendations. Through the case study, we find that the SMURF methodology serves as an effective engagement tool to get developers, users, and product owners interested in sustainable software ideas.
... Read more

Containerization with Docker has become the standard for the deployment of software in recent years since it is a lightweight method to isolate applications. However, the selection of a Docker image brings different dependencies, which can introduce energy inefficiencies that are often not documented. Recent work performs a comparative study between different base images and finds certain patterns in which some images tend to be less energy efficient for certain tasks. In this thesis, we propose a methodology to find the reason behind observed energy inefficiencies in a Docker image by using tracing to study how a workload makes use of the dependencies provided by the image. We apply this methodology to test the energy efficiency of Redis when using different base images. This leads to finding and raising awareness of energy inefficiencies in some functions from musl, the C standard library implementation used by Alpine. ... Read more

As data centers worldwide consume more power than ever, lowering the energy consumption of software is increasingly important. Software energy testing is often unclear due to a lack of comparable baselines. In this paper, we look at the use of regression testing to alleviate some of the struggles with energy testing. We introduce E-Compare, a tool designed to identify energy regressions in software updates by comparing the energy consumption of different versions of the same project. E-Compare is cloud-based, fully automated, and can be implemented in any project with just three lines of code. To validate its effectiveness, we applied E-Compare to thirteen real-world projects, ranging from long-established projects to newer, active ones. Over 700 code changes have been tested. Our findings indicate that energy regression testing can identify energy regressions missed by developers. Some of the indicated energy regressions could be traced back to specific code changes, confirming the tool’s accuracy and relevance. However, the tool’s usability varies significantly depending on the project, and unexpected energy regressions are relatively rare. ... Read more

Edge AI is an architectural deployment tactic that brings AI models closer to the user and data, relieving internet bandwidth usage and providing low latency and privacy. It remains unclear how this tactic performs at scale, since the distribution overhead could impact the total energy consumption. We identify four architectural scalability factors that could impact the energy consumption of AI: environment, optimisation, throughput, and overhead. The latter consists of downloading, verification, and updating the model over time. This work performs an empirical study on the sustainability of Edge AI compared to Cloud AI at scale in terms of energy consumption. For the environment variable, energy consumption measurement experiments are run on a cloud device and multiple edge devices, various quantized models for optimisation, and various throughput levels per hour. We simulate the distribution overhead and combine the results with the measurements to find the holistic energy efficiency of each architectural strategy. We find that all four variables impact energy consumption, but the main contributors are environment, throughput, and overhead. We observe that Edge AI is most energy-efficient in low-distribution, low-demand scenarios, whereas in high-distribution, high-demand scenarios Cloud AI is better optimised and outperforms Edge AI in energy efficiency. This means that developers depending on their use case and the project’s scalability need to consider these quality attributes for the most sustainable architectural solution. ... Read more

In Sustainable Software Engineering there is a need for tooling and guidelines for developers. In this research we aim to provide such guidelines. We find that for our experimental setup and set of benchmarks 500 samples gives results that are likely stable at a 1% threshold in their Relative Confidence Interval Width. Running benchmarks with a variable CPU clock-speed can lead to higher variability of measurements; as well as initialising benchmarks with random data. Likewise we investigate the effect of the length of benchmarks on their stability but we can not rule out that this is caused by the experiment setup. Lastly we identify control flow statements and code related to memory accesses as potential large influences of instability. ... Read more

Continuous Integration (CI) has become a cornerstone of modern software development, gaining widespread adoption due to its ability to facilitate frequent and dependable code integration. However, its benefits are offset by high computational costs and energy consumption, particularly in the build phase. With its growing popularity, it is crucial to reflect on the efficiency of the CI process. This thesis proposes a novel framework to optimise energy consumption in the build jobs of CI pipelines, with primary focus on minimising compilation workload. Leveraging static dependency analysis and commit information, the framework introduces guided partial compilation, targeting only files affected by changes. The results demonstrate its ability to maintain CI reliability while significantly reducing energy consumption in real-world projects, with a 22% reduction of energy consumption in compilation-only experiments, and up to 63% energy savings in experiments that extrapolate the effects of partial compilation across the rest of the build job. The contributions in this research offer a stepping stone toward the imperative establishment of sustainable standards within the CI practice. ... Read more

In large-scale ML, data size becomes a critical variable, especially in the context of large companies, where models already exist and are hard to change and fine-tune. Time to market and model quality are essential metrics, thus looking for ways to select, prune and augment the input data while treating the model as a black box can speed up the process from raw data to productionized model.

Datasets can have thousands of features and many redundant/duplicate samples, for various business logic reasons. In some particular ML flows, it might be that only a subset of them provide most of the input to the final accuracy. Also, looking into ways to provide insights on what data points are the most meaningful can help engineers collect more relevant samples, or focus their attention on specific parts of the data distribution.
... Read more

The internet is a system that was introduced over 30 years ago and has taken over the world since then. Connecting over 5 billion people is possible thanks to the global scale that the internet operates at. While the advantages are unmistakable, we must also acknowledge that the internet is a large contributor to the emission of greenhouse gasses. The footprint of the individual user might be relatively small, but as the number of users is only expected to grow, we must find a way to make the internet more sustainable. In this thesis we explore the state of sustainable web design from three different perspectives: academic, development and end user. Based on academic publications we create a catalogue of nine green web patterns that have empirically been shown to reduce the energy consumption. Secondly, we analyse developer communication on GitHub to see what green patterns are being used in practice. Our results show that there is very limited conversation about the implementation of green patterns, whereas for mobile app development the use of green patterns is more common. Lastly, we take on the end user's perspective and audit the websites of universities to find a relation between a sustainable reputation and a sustainable website. For the audition of websites we create a Google Lighthouse plugin specifically to test sustainability. We find a significant difference between the websites from the group of most sustainable universities and the group of least sustainable universities. The main takeaway of this thesis is that there is a lot of room for improvement to make websites more sustainable. A lack of awareness found from all three perspectives is currently a bottleneck for wider adoption of green patterns. We argue that the availability of development tools with built-in sustainability features could increase awareness and adoption of sustainable web design. ... Read more

Integrating Artificial Intelligence (AI) into software systems has significantly enhanced their capabilities while escalating energy demands. Ensemble learning, combining predictions from multiple models to form a single prediction, intensifies this problem due to cumulative energy consumption.

This paper presents a novel approach to model selection that addresses the challenge of balancing the accuracy of AI models with their energy consumption in a live AI ensemble system. We explore how reducing the number of models or improving the efficiency of model usage within an ensemble during inference can reduce energy demands without substantially sacrificing accuracy.

This study introduces and evaluates two model selection strategies, Static and Dynamic, for optimizing ensemble learning systems' performance while minimizing energy usage. Our results demonstrate that the Static strategy improves the F1 score beyond the baseline, reducing average energy usage from 100% from the full ensemble to 62%.
The Dynamic strategy further enhances F1 scores, while using on average 76% compared to 100% of the full ensemble.

Moreover, we propose an approach that balances accuracy with resource consumption, significantly reducing energy usage without substantially impacting accuracy. This method decreased the average energy usage of the Static strategy from approximately 62% to 14%, and for the Dynamic strategy, from around 76% to 57%.

Our field study of Green AI using an operational AI system developed by a large professional services provider shows the practical applicability of adopting energy-conscious model selection strategies in live production environments. ... Read more

Over the past years the size of deep learning models has been growing consistently. This growth has led to significant improvements in performance, but at the expense of increased computational resource demands. Compression techniques can be used to improve the efficiency of deep learning models by shrinking their size and computational needs, while
preserving performance.


This thesis presents EasyCompress, an automated and user-friendly tool to compress deep learning models. The tool improves on existing compression research by focusing on generalizability and practical usability, in three ways. Firstly, it aligns with specific compression objectives and performance requirements, ensuring the compression accomplishes its intended goal effectively. Secondly, it employs flexible compression techniques, so that it is applicable to a diverse set of models without requiring deep model knowledge. Finally, it automates the compression process, eliminating difficult and time-consuming implementation
efforts.


EasyCompress intelligently selects, tailors, and combines various compression techniques to minimize model size, latency, or number of computations while preserving performance. It employs structured pruning to reduce the number of parameters and computations, uses knowledge distillation techniques to ensure better accuracy recovery, and uses quantization to achieve additional compression.


The tool’s effectiveness is evaluated across diverse model architectures and configurations. Experimental results on a range of models and datasets demonstrate its ability to reduce the model size at least 5-fold, inference time by at least 1.5-fold, and the number of computations by at least 3-fold. Most compression rates are even higher, reaching up to 10, 20, and even 100-fold reductions.


The tool is available online at https://thesis.abelvansteenweghen.com. ... Read more

This thesis was written in during my internship at Adyen as the final project of the Master’s program in Computer Science at the TU Delft. In my Bachelor’s thesis, I compared the energy consumption of three Android UI frameworks, and I chose to continue working on the subject of energy consumption on Android as it is relevant both to Adyen and the greater Android community. In this thesis, I review the state-of-the-art of energy consumption tools for Android, compare three approaches to attributing energy consumption to code at a fine-grained level, and implement one of these in a tool that can be used to analyze release-mode Android apps. I then empirically evaluate this tool, and apply it at Adyen in a case study in which I cooperate with a developer to solve an energy bug. As part of the empirical evaluation, I also perform a preliminary comparison of release and debug mode with respect to energy consumption, using three code smells identified in prior work as having an effect on the energy use of methods containing them. The results of the empirical investigation show that statistical sampling can be applied to Android devices to attribute energy consumption to methods within a reasonable margin of error. It further shows that this approach can be used to identify differences in energy consumption between different versions of software. The comparison between release and debug mode showed that the overhead caused by the use of debug mode is not consistent, and varies between both different code smells and different devices. This has significant implications for further work measuring energy consumption on Android, as it implies that results obtained using debug mode cannot be generalized to release mode, which all apps will use in production environments. Finally, the case study showed that this approach is able to significantly assist the energy debugging process, and revealed that even today, developers often lack the information necessary to make informed decisions over the energy consumption of their software. However, once this information is made available, both developers and stakeholders are willing to adapt their decision-making to incorporate energy efficiency. ... Read more

Advances in data science have caused an increase in the use of Artificial Intelligence (AI), specifically Machine Learning (ML), throughout various fields. Not only in research but in the industry as well, has ML been receiving increasing amounts of interest. Many companies rely on ML models to increase the efficiency of existing processes or offer new services and products. The industry, however, is facing several additional challenges compared to the academic context. One of those challenges is applying the Development Operations (DevOps) model to an ML application, also referred to as MLOps. This thesis sets out to find the specific challenges that practitioners encounter while operationalising ML models. To do so, we perform a single-case case study on an ML pipeline built by the Trade & Communication Surveillance team at the ING bank. This case study consists of conducting a set of interviews and performing a manual code inspection of the pipeline. The team faces challenges ranging from having insufficient time for operationalising each ML project individually to operating in the highlyregulated fintech context. Their pipeline is able to deploy a single ML model but it does not generalise well to other projects. We present the first version of an application that mitigates these challenges. The application is able to deploy ML models to the development environment at ING and can be operated by data scientists to reduce the effort of operationalising an ML model. ... Read more

Dependency management is an important task in software maintenance. However, identifying and removing unused dependencies takes a lot of effort from developers as existing tools may discover many false positives which are challenging to distinguish. This paper proposes a decision framework to improve unused dependency detection. It is applied to an industrial Maven project. Firstly, OPAL(a call graph tool) augments the call graph of a dependency analysis tool DepClean to support dynamic features of Java. Secondly, the classification of the relationship between dependencies simplifies the comprehension of an unused dependency. Thirdly, a decision process prioritizes the test of removing unnecessary dependencies. Results show that developers can focus their efforts on maintaining bloated dependencies by following the recommendation of the proposed decision process. It is particularly noteworthy that this decision framework helps reduce one-third of false positives of unused dependencies in a given industrial Maven project. In addition, our suggestions are compared to the motive of removing dependencies in three open-source Maven projects. Results indicate that our advice is consistent in the reasoning behind removing dependencies. Hence, this work reduces the effort for developers to decide on dependency elimination. ... Read more

The popularity of machine learning has wildly expanded in recent years. Machine learning techniques have been heatedly studied in academia and applied in the industry to create business value. However, there is a lack of guidelines for code quality in machine learning applications. Although machine learning code is usually integrated as a small part of an overarching system, it usually plays an important role in its core functionality. Hence ensuring code quality is quintessential to avoiding issues in the long run. To help improve the machine learning code quality, we conducted two studies in this thesis. The first study proposes and identifies a list of 22 machine learning-specific code smells collected from various sources, including papers, grey literature, GitHub commits, and Stack Overflow posts. We pinpoint each smell with a description of its context, potential issues in the long run, and proposed solutions. In addition, we link them to their respective pipeline stage and the evidence from both academic and grey literature. The second study aims to develop a tool to improve code quality and study the prevalence of machine learning-specific code smells. We extend a static analysis tool dslinter and run it on both Python notebook datasets and regular Python project datasets. Moreover, we analyse the result to check the tool's validity and investigate the code smell prevalence in machine learning applications. The code smell catalog and dslinter together help data scientists and developers produce and maintain high-quality machine learning application code. ... Read more

Microservice architectures (MSA) have become a dominant architectural style choice in the service oriented software industry. Because of this, as with any other system, some unoptimized approaches might creep into architectures. These are what we call anti-patterns, they can be considered the opposite of design patterns. Furthermore, a microservice architecture can quickly grow to an immense scale due to the number of services.

In this work, we present Luduan, a tool created within ING that provides engineers with insights into their MSA. Using different graph metrics and tracing data, we determine the likelihood of any service containing certain anti-patterns. We validate this methodology by gathering feedback from subject-matter experts, by ways of surveys and one-on-one sessions where Luduan is used as a support tool. Finally, we ask teams that are responsible for services, to figure out whether their service has an anti-pattern or not. We then use these results to fine tune the computations from metrics to anti-pattern likelihood. ... Read more

With the advancement of technology, organizations are experiencing more trouble with keeping their data private with it often leaked to the public via their code-repositories or databases. There are methods to counter the leakage of data while pushing code to a repository however, these are heavily reliant on regular expressions. Personal names, locations and other Personally Identifiable Information (PII) do not follow a reoccurring pattern and can thus only be prevented by manual code reviews, which are also prone to errors. A tool to detect these PII should be designed as an initial measure to counteract the leakage. In this paper, we propose a heavily modifiable tool in which we combine the strength of regular expressions with a state-of-the-art machine learning model to detect a variety of important PII within the code changes of Python software projects. We use CodeBERT, a RoBERTa-like Transformer model, as our PII recognizer. This recognizer is fine-tuned using the Scikit-learn library of which we injected the git commits with fake sensitive data. To test and improve the quality of the model and the entire tool, we design an experimental methodology to find the optimal value for the hyper parameters of the model, compare it against another Transformer model and run the fine-tuned model against several other code-bases with different programming languages. The outcome of these experiments benefit the quality of the model in a positive way and allows us to design a robust tool with a well-performing machine learning model to detect a variety of entities. This tool can be personalized to any business and mitigate a significant part of the potential data leaks. ... Read more

In this work, we look at the intersection of Sustainable Software Engineering and AI engineering known as Green AI. AI computing is rapidly becoming more expensive, calling for a change in design philosophy. We consider both training and inference of neural networks used for image vision; to reveal energy-efficient practices in an exploratory fashion.

First of all, we examine a modern algorithm for hyperparameter optimisation and compare this to two baseline methods. We find that the baseline algorithms perform considerably worse despite their wide usage and argue that they should not be used when training large models. Furthermore, we look at the layer structure of convolutional networks and conclude that the convolutional layers have the largest influence on the total consumption. We report increases of up to 95% with only marginal improvements in accuracy. Therefore we recommend developers to reduce their network architectures as long as the performance stays within a reasonable margin.

Second, we present a study focused on the inference phase of the deep learning pipeline. We look at the effect of batching for image classification requests. To facilitate the data collection, we make use of a simulated queue and the Pytorch framework. We find that batching has a significant impact on the energy consumption, but the magnitude of this impact can vary a lot for different models. Our recommendation is to treat the batch size as an inference parameter that needs to be tuned first. Additionally, we highlight how the energy consumption of image vision networks has evolved over the past decade. Presenting the findings together with the performance of these networks shows a steady, upward energy trend accompanied by a decreasing slope for the accuracy. The only exception is the model ShuffleNetV2. We mention the design principles that went into the development of this network and present it as a start for future research. ... Read more