Measuring up to Stability
Guidelines towards accurate energy consumption measurement results of Rust benchmarks
R. Hijdra (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Luis Cruz – Mentor (TU Delft - Software Engineering)
Arie Van Van Deursen – Graduation committee member (TU Delft - Software Engineering)
Christoph Laaber – Coach (Simula.no)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
In Sustainable Software Engineering there is a need for tooling and guidelines for developers. In this research we aim to provide such guidelines. We find that for our experimental setup and set of benchmarks 500 samples gives results that are likely stable at a 1% threshold in their Relative Confidence Interval Width. Running benchmarks with a variable CPU clock-speed can lead to higher variability of measurements; as well as initialising benchmarks with random data. Likewise we investigate the effect of the length of benchmarks on their stability but we can not rule out that this is caused by the experiment setup. Lastly we identify control flow statements and code related to memory accesses as potential large influences of instability.