JC
J.M. Chan
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
The introduction of large language models (LLMs) has transformed the way software is written. With the help of LLM powered code generation the productivity of software engineers has increased all over the world. However, these models are also computationally expensive. The ubiquitous use of these models has raised significant sustainability concerns.
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial. ...
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial. ...
The introduction of large language models (LLMs) has transformed the way software is written. With the help of LLM powered code generation the productivity of software engineers has increased all over the world. However, these models are also computationally expensive. The ubiquitous use of these models has raised significant sustainability concerns.
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial.
LLM routing aims to reduce the usage of more complex models by routing easier tasks to smaller models. However, existing research on routing primarily focuses on monetary savings and the potential for routing from a sustainability perspective has yet to be explored.
In this thesis we propose an energy-aware LLM routing framework to measure, train and evaluate various routers. We implement our framework and conduct experiments to quantify the energy efficiency of routing and to examine the trade-offs between accuracy and energy consumption. Furthermore, we analyze the overhead introduced by the various routing components. Our results show that routing can reduce energy consumption by up to 15.3\% on the HumanEval and MBPP dataset with minimal overhead when compared to a interpolated baseline. However, overall energy savings were found to decrease significantly as we aim for accuracy targets near the stronger model. These findings show that LLM routing is a viable strategy to reduce energy consumption of LLM code generation in scenarios where achieving maximum performance is not crucial.
The ability to accurately determine the location within indoor settings is crucial for various applications such as indoor navigation, interactive floor plans, and room-specific services. While GPS technology has revolutionized outdoor positioning, it falls short in providing precise location information within buildings due to signal blockage. To address this limitation, specialized indoor positioning systems utilizing acoustic sensing have been explored, leveraging deep learning models. This paper presents a comparative study of passive and active acoustic sensing systems for room recognition. Passive sensing involves capturing existing background noise in a room and using it as an identifier, while active sensing emits acoustic signals and analyzes the resulting echoes. Previous research has primarily focused on active sensing, achieving high classification accuracy but facing challenges related to device orientation and the presence of multiple individuals. Moreover, the emission of high-frequency chirps used in active sensing may cause discomfort to pets. The results indicate that passive sensing achieves an accuracy of 73.7%, slightly outperforming active sensing at 63.5% in baseline conditions. However, in the presence of constant background noise, passive sensing accuracy drops to 21.7%, while active sensing exhibits better resilience with an accuracy of 59.7%. Furthermore, when the device orientation is altered by 90 degrees, active sensing results in a lower in accuracy (45.5%), while passive sensing maintains better performance at 71%. The impact of multiple individuals in the room had a relatively minor effect on passive sensing systems, achieving an accuracy of 72.2%. Active sensing was shown to be not as resilient, reaching an accuracy of 44.2%.
...
...
The ability to accurately determine the location within indoor settings is crucial for various applications such as indoor navigation, interactive floor plans, and room-specific services. While GPS technology has revolutionized outdoor positioning, it falls short in providing precise location information within buildings due to signal blockage. To address this limitation, specialized indoor positioning systems utilizing acoustic sensing have been explored, leveraging deep learning models. This paper presents a comparative study of passive and active acoustic sensing systems for room recognition. Passive sensing involves capturing existing background noise in a room and using it as an identifier, while active sensing emits acoustic signals and analyzes the resulting echoes. Previous research has primarily focused on active sensing, achieving high classification accuracy but facing challenges related to device orientation and the presence of multiple individuals. Moreover, the emission of high-frequency chirps used in active sensing may cause discomfort to pets. The results indicate that passive sensing achieves an accuracy of 73.7%, slightly outperforming active sensing at 63.5% in baseline conditions. However, in the presence of constant background noise, passive sensing accuracy drops to 21.7%, while active sensing exhibits better resilience with an accuracy of 59.7%. Furthermore, when the device orientation is altered by 90 degrees, active sensing results in a lower in accuracy (45.5%), while passive sensing maintains better performance at 71%. The impact of multiple individuals in the room had a relatively minor effect on passive sensing systems, achieving an accuracy of 72.2%. Active sensing was shown to be not as resilient, reaching an accuracy of 44.2%.