TY
T.E.R. Yarally
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
3 records found
1
The batch size is an essential parameter to tune during the development of new neural networks. Amongst other quality indicators, it has a large degree of influence on the model’s accuracy, generalisability, training times and parallelisability. This fact is generally known and commonly studied. However, during the application phase of a deep learning model, when the model is utilised by an end-user for inference, we find that there is a disregard for the potential benefits of introducing a batch size. In this study, we examine the effect of input batching on the energy consumption and response times of five fully-trained neural networks for computer vision that were considered state-of-the-art at the time of their publication. The results suggest that batching has a significant effect on both of these metrics. Furthermore, we present a timeline of the energy efficiency and accuracy of neural networks over the past decade. We find that in general, energy consumption rises at a much steeper pace than accuracy and question the necessity of this evolution. Additionally, we highlight one particular network, ShuffleNetV2 (2018), that achieved a competitive performance for its time while maintaining a much lower energy consumption. Nevertheless, we highlight that the results are model dependent.
...
The batch size is an essential parameter to tune during the development of new neural networks. Amongst other quality indicators, it has a large degree of influence on the model’s accuracy, generalisability, training times and parallelisability. This fact is generally known and commonly studied. However, during the application phase of a deep learning model, when the model is utilised by an end-user for inference, we find that there is a disregard for the potential benefits of introducing a batch size. In this study, we examine the effect of input batching on the energy consumption and response times of five fully-trained neural networks for computer vision that were considered state-of-the-art at the time of their publication. The results suggest that batching has a significant effect on both of these metrics. Furthermore, we present a timeline of the energy efficiency and accuracy of neural networks over the past decade. We find that in general, energy consumption rises at a much steeper pace than accuracy and question the necessity of this evolution. Additionally, we highlight one particular network, ShuffleNetV2 (2018), that achieved a competitive performance for its time while maintaining a much lower energy consumption. Nevertheless, we highlight that the results are model dependent.
Green AI
An empirical study
Master thesis
(2022)
-
T.E.R. Yarally, A. van Deursen, L. Miranda da Cruz, M. Weinmann, Daniel Feitosa
In this work, we look at the intersection of Sustainable Software Engineering and AI engineering known as Green AI. AI computing is rapidly becoming more expensive, calling for a change in design philosophy. We consider both training and inference of neural networks used for image vision; to reveal energy-efficient practices in an exploratory fashion.
First of all, we examine a modern algorithm for hyperparameter optimisation and compare this to two baseline methods. We find that the baseline algorithms perform considerably worse despite their wide usage and argue that they should not be used when training large models. Furthermore, we look at the layer structure of convolutional networks and conclude that the convolutional layers have the largest influence on the total consumption. We report increases of up to 95% with only marginal improvements in accuracy. Therefore we recommend developers to reduce their network architectures as long as the performance stays within a reasonable margin.
Second, we present a study focused on the inference phase of the deep learning pipeline. We look at the effect of batching for image classification requests. To facilitate the data collection, we make use of a simulated queue and the Pytorch framework. We find that batching has a significant impact on the energy consumption, but the magnitude of this impact can vary a lot for different models. Our recommendation is to treat the batch size as an inference parameter that needs to be tuned first. Additionally, we highlight how the energy consumption of image vision networks has evolved over the past decade. Presenting the findings together with the performance of these networks shows a steady, upward energy trend accompanied by a decreasing slope for the accuracy. The only exception is the model ShuffleNetV2. We mention the design principles that went into the development of this network and present it as a start for future research. ...
First of all, we examine a modern algorithm for hyperparameter optimisation and compare this to two baseline methods. We find that the baseline algorithms perform considerably worse despite their wide usage and argue that they should not be used when training large models. Furthermore, we look at the layer structure of convolutional networks and conclude that the convolutional layers have the largest influence on the total consumption. We report increases of up to 95% with only marginal improvements in accuracy. Therefore we recommend developers to reduce their network architectures as long as the performance stays within a reasonable margin.
Second, we present a study focused on the inference phase of the deep learning pipeline. We look at the effect of batching for image classification requests. To facilitate the data collection, we make use of a simulated queue and the Pytorch framework. We find that batching has a significant impact on the energy consumption, but the magnitude of this impact can vary a lot for different models. Our recommendation is to treat the batch size as an inference parameter that needs to be tuned first. Additionally, we highlight how the energy consumption of image vision networks has evolved over the past decade. Presenting the findings together with the performance of these networks shows a steady, upward energy trend accompanied by a decreasing slope for the accuracy. The only exception is the model ShuffleNetV2. We mention the design principles that went into the development of this network and present it as a start for future research. ...
In this work, we look at the intersection of Sustainable Software Engineering and AI engineering known as Green AI. AI computing is rapidly becoming more expensive, calling for a change in design philosophy. We consider both training and inference of neural networks used for image vision; to reveal energy-efficient practices in an exploratory fashion.
First of all, we examine a modern algorithm for hyperparameter optimisation and compare this to two baseline methods. We find that the baseline algorithms perform considerably worse despite their wide usage and argue that they should not be used when training large models. Furthermore, we look at the layer structure of convolutional networks and conclude that the convolutional layers have the largest influence on the total consumption. We report increases of up to 95% with only marginal improvements in accuracy. Therefore we recommend developers to reduce their network architectures as long as the performance stays within a reasonable margin.
Second, we present a study focused on the inference phase of the deep learning pipeline. We look at the effect of batching for image classification requests. To facilitate the data collection, we make use of a simulated queue and the Pytorch framework. We find that batching has a significant impact on the energy consumption, but the magnitude of this impact can vary a lot for different models. Our recommendation is to treat the batch size as an inference parameter that needs to be tuned first. Additionally, we highlight how the energy consumption of image vision networks has evolved over the past decade. Presenting the findings together with the performance of these networks shows a steady, upward energy trend accompanied by a decreasing slope for the accuracy. The only exception is the model ShuffleNetV2. We mention the design principles that went into the development of this network and present it as a start for future research.
First of all, we examine a modern algorithm for hyperparameter optimisation and compare this to two baseline methods. We find that the baseline algorithms perform considerably worse despite their wide usage and argue that they should not be used when training large models. Furthermore, we look at the layer structure of convolutional networks and conclude that the convolutional layers have the largest influence on the total consumption. We report increases of up to 95% with only marginal improvements in accuracy. Therefore we recommend developers to reduce their network architectures as long as the performance stays within a reasonable margin.
Second, we present a study focused on the inference phase of the deep learning pipeline. We look at the effect of batching for image classification requests. To facilitate the data collection, we make use of a simulated queue and the Pytorch framework. We find that batching has a significant impact on the energy consumption, but the magnitude of this impact can vary a lot for different models. Our recommendation is to treat the batch size as an inference parameter that needs to be tuned first. Additionally, we highlight how the energy consumption of image vision networks has evolved over the past decade. Presenting the findings together with the performance of these networks shows a steady, upward energy trend accompanied by a decreasing slope for the accuracy. The only exception is the model ShuffleNetV2. We mention the design principles that went into the development of this network and present it as a start for future research.
Bachelor thesis
(2020)
-
Tim Yarally, Toby van Willegen, Mees Brinkhuis, Dirk den Hoedt, Mike van der Meer, J. Urbano Merino, Otto Visser, Thomas Overklift Vaupel Klein
Items being misplaced in warehouses easily get lost. To combat this, warehouses have to send people in scanning all barcodes in the warehouse. This is highly inefficient, which is why Eonics wants to build a drone handling this. There are options out there to scan barcodes, but none of them match the requirements laid out by Eonics. Among these requirements are a lightweight camera, such as a GoPro, and a recording distance of 1.5-2 metres. This report will look and see if these requirements are feasible. Techniques used in this report are Mathematical Morphology, Maximally Stable Extremal Regions, Convolutional Neural Networks, Gradiental Difference and Direction Estimation with Region Extraction. The report concludes in stating that interpreting the barcodes is not possible with mere software under these requirements. The maximal distance we were able to interpret barcodes from, based on a 4K image, was around 1 metre. Continuing the trend, we would need at least an 8K camera to detect from a distance of 1.5 metres. Detection however, is less difficult and is feasible from a distance of 1.5-2 metres. The report also derives an function to use to calculate the maximum distance a barcode can be interpreted from, based on the details of the barcode and camera. Finally, research is done regarding using hardware solutions, such as a zoom-lens, which has promising results.
...
Items being misplaced in warehouses easily get lost. To combat this, warehouses have to send people in scanning all barcodes in the warehouse. This is highly inefficient, which is why Eonics wants to build a drone handling this. There are options out there to scan barcodes, but none of them match the requirements laid out by Eonics. Among these requirements are a lightweight camera, such as a GoPro, and a recording distance of 1.5-2 metres. This report will look and see if these requirements are feasible. Techniques used in this report are Mathematical Morphology, Maximally Stable Extremal Regions, Convolutional Neural Networks, Gradiental Difference and Direction Estimation with Region Extraction. The report concludes in stating that interpreting the barcodes is not possible with mere software under these requirements. The maximal distance we were able to interpret barcodes from, based on a 4K image, was around 1 metre. Continuing the trend, we would need at least an 8K camera to detect from a distance of 1.5 metres. Detection however, is less difficult and is feasible from a distance of 1.5-2 metres. The report also derives an function to use to calculate the maximum distance a barcode can be interpreted from, based on the details of the barcode and camera. Finally, research is done regarding using hardware solutions, such as a zoom-lens, which has promising results.