TN
T.J. Nulle
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
This thesis investigates reducing carbon emissions in code generation using large language models (LLMs) by comparing function-level and line-level code completions across models of different sizes (1.5B and 9B parameters). The study utilises the BigCodeBench dataset, comprising 1,140 Python programming problems, to evaluate the energy consumption, test accuracy, and time efficiency of code completions. The models, 4-bit quantised and run on a CPU, performed 30 function-level completions and 30 line-level completions for each line, which were tested for correctness. Results indicate that, while line-level completions require slightly more energy per token, they are more efficient overall in terms of total energy consumption and token usage. The smaller model with line-level completions showed significant reductions in carbon emissions, achieving an average tenfold reduction compared to the large model with function-level completions. With the large model, line-level completions achieved a $4.5\times$ reduction in carbon emissions compared to function-level completions. Line-level completions were more token-efficient, wasting less than 1\% of energy, compared to 20\% for function-level completions. From a sustainability perspective, line-level completions offer a practical strategy to reduce the environmental impact of code generation tasks while maintaining strong performance. The study suggests that optimising completion strategies could help balance energy consumption, test accuracy, and time efficiency. Future research could explore a broader range of model sizes, fine-tuning models specifically for line-level completions, a performance decrease in solution length, and alternative validation metrics to assess code generation performance.
...
This thesis investigates reducing carbon emissions in code generation using large language models (LLMs) by comparing function-level and line-level code completions across models of different sizes (1.5B and 9B parameters). The study utilises the BigCodeBench dataset, comprising 1,140 Python programming problems, to evaluate the energy consumption, test accuracy, and time efficiency of code completions. The models, 4-bit quantised and run on a CPU, performed 30 function-level completions and 30 line-level completions for each line, which were tested for correctness. Results indicate that, while line-level completions require slightly more energy per token, they are more efficient overall in terms of total energy consumption and token usage. The smaller model with line-level completions showed significant reductions in carbon emissions, achieving an average tenfold reduction compared to the large model with function-level completions. With the large model, line-level completions achieved a $4.5\times$ reduction in carbon emissions compared to function-level completions. Line-level completions were more token-efficient, wasting less than 1\% of energy, compared to 20\% for function-level completions. From a sustainability perspective, line-level completions offer a practical strategy to reduce the environmental impact of code generation tasks while maintaining strong performance. The study suggests that optimising completion strategies could help balance energy consumption, test accuracy, and time efficiency. Future research could explore a broader range of model sizes, fine-tuning models specifically for line-level completions, a performance decrease in solution length, and alternative validation metrics to assess code generation performance.
Even though previous studies have studied software artefacts on a package level, little research has been done on a method level. In this work, we perform a method-level analysis to determine how popularity disperses among methods within software libraries of Maven Central. We analyse 384 software artefacts with three different metrics: eigenvector centrality, degree centrality and dependent usage percentage. Using callgraphs of the interactions of a software artefact with its dependents, we can determine the relative popularity score of any method. We observe that popularity is inverse logarithmically distributed among the most frequently used methods within a library. Furthermore, 80% of calls to a library are to 26% of all methods, following the Pareto Principle. Likewise, the number of dependents per artefacts also follows a power-law distribution. We also find that no significant correlation exists between any of the analysed metrics, allowing opportunities for future research to determine a more accurate popularity metric. All of our results show that method popularity is logarithmically distributed within software artefacts of Maven Central.
...
...
Even though previous studies have studied software artefacts on a package level, little research has been done on a method level. In this work, we perform a method-level analysis to determine how popularity disperses among methods within software libraries of Maven Central. We analyse 384 software artefacts with three different metrics: eigenvector centrality, degree centrality and dependent usage percentage. Using callgraphs of the interactions of a software artefact with its dependents, we can determine the relative popularity score of any method. We observe that popularity is inverse logarithmically distributed among the most frequently used methods within a library. Furthermore, 80% of calls to a library are to 26% of all methods, following the Pareto Principle. Likewise, the number of dependents per artefacts also follows a power-law distribution. We also find that no significant correlation exists between any of the analysed metrics, allowing opportunities for future research to determine a more accurate popularity metric. All of our results show that method popularity is logarithmically distributed within software artefacts of Maven Central.