The Speedup‐Test: a statistical methodology for programme speedup analysis and computation. (15th October 2012)
- Record Type:
- Journal Article
- Title:
- The Speedup‐Test: a statistical methodology for programme speedup analysis and computation. (15th October 2012)
- Main Title:
- The Speedup‐Test: a statistical methodology for programme speedup analysis and computation
- Authors:
- Touati, Sid‐Ahmed‐Ali
Worms, Julien
Briais, Sébastien
Smari, Waleed
Fiore, Sandro
Hill, David - Abstract:
- <abstract abstract-type="main" id="cpe2939-abs-0001"> <title>SUMMARY</title> <p id="cpe2939-para-0001">In the area of high‐performance computing and embedded systems, numerous code optimisation methods exist to accelerate the speed of the computation (or optimise another performance criteria). They are usually experimented by doing multiple observations of the initial and the optimised execution times of a programme in order to declare a speedup. Even with fixed input and execution environment, programme execution times vary in general. Hence, different kinds of speedups may be reported: the speedup of the average execution time, the speedup of the minimal execution time, the speedup of the median and others. Many published speedups in the literature are observations of a set of experiments. To improve the reproducibility of the experimental results, this article presents a rigorous statistical methodology regarding programme performance analysis. We rely on well‐known statistical tests (Shapiro–Wilk's test, Fisher's <italic>F</italic>‐test, Student's <italic>t</italic>‐test, Kolmogorov–Smirnov's test and Wilcoxon–Mann–Whitney's test) to study if the observed speedups are statistically significant or not. By fixing 0 &lt; <italic>α</italic> &lt; 1 a desired risk level, we are able to analyse the statistical significance of the average execution time as well as the median. We can also check if <inline-graphic mimetype="image" xlink:href="ark:/27927/pgg2248xjwb"<abstract abstract-type="main" id="cpe2939-abs-0001"> <title>SUMMARY</title> <p id="cpe2939-para-0001">In the area of high‐performance computing and embedded systems, numerous code optimisation methods exist to accelerate the speed of the computation (or optimise another performance criteria). They are usually experimented by doing multiple observations of the initial and the optimised execution times of a programme in order to declare a speedup. Even with fixed input and execution environment, programme execution times vary in general. Hence, different kinds of speedups may be reported: the speedup of the average execution time, the speedup of the minimal execution time, the speedup of the median and others. Many published speedups in the literature are observations of a set of experiments. To improve the reproducibility of the experimental results, this article presents a rigorous statistical methodology regarding programme performance analysis. We rely on well‐known statistical tests (Shapiro–Wilk's test, Fisher's <italic>F</italic>‐test, Student's <italic>t</italic>‐test, Kolmogorov–Smirnov's test and Wilcoxon–Mann–Whitney's test) to study if the observed speedups are statistically significant or not. By fixing 0 &lt; <italic>α</italic> &lt; 1 a desired risk level, we are able to analyse the statistical significance of the average execution time as well as the median. We can also check if <inline-graphic mimetype="image" xlink:href="ark:/27927/pgg2248xjwb" xlink:type="simple" xmlns:xlink="http://www.w3.org/1999/xlink" /><mml:math display="block" altimg="urn:x-wiley:15320626:media:cpe2939:cpe2939-math-0001" overflow="scroll" xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi mathvariant="double-struck">P</mml:mi><mml:mfenced open="[" close="]"><mml:mrow><mml:mi>X</mml:mi><mml:mo class="MathClass-rel">&gt;</mml:mo><mml:mi>Y</mml:mi></mml:mrow></mml:mfenced><mml:mo class="MathClass-rel">&gt;</mml:mo><mml:mn>1</mml:mn><mml:mo class="MathClass-bin">∕</mml:mo><mml:mn>2</mml:mn></mml:math>, the probability that an individual execution of the optimised code is faster than the individual execution of the initial code. In addition, we can compute the confidence interval of the probability to obtain a speedup on a randomly selected benchmark that does not belong to the initial set of tested benchmarks. Our methodology defines a consistent improvement compared with the usual performance analysis method in high‐performance computing. We explain in each situation the hypothesis that must be checked to declare a correct risk level for the statistics. The Speedup‐Test protocol certifying the observed speedups with rigorous statistics is implemented and distributed as an open source tool based on <monospace>R</monospace> software. Copyright © 2012 John Wiley &amp; Sons, Ltd.</p> </abstract> … (more)
- Is Part Of:
- Concurrency and computation. Volume 25:Number 10(2013:Jul.)
- Journal:
- Concurrency and computation
- Issue:
- Volume 25:Number 10(2013:Jul.)
- Issue Display:
- Volume 25, Issue 10 (2013)
- Year:
- 2013
- Volume:
- 25
- Issue:
- 10
- Issue Sort Value:
- 2013-0025-0010-0000
- Page Start:
- 1410
- Page End:
- 1426
- Publication Date:
- 2012-10-15
- Subjects:
- Parallel processing (Electronic computers) -- Periodicals
Parallel computers -- Periodicals
004.35 - Journal URLs:
- http://onlinelibrary.wiley.com/ ↗
- DOI:
- 10.1002/cpe.2939 ↗
- Languages:
- English
- ISSNs:
- 1532-0626
- Deposit Type:
- Legaldeposit
- View Content:
- Available online (eLD content is only available in our Reading Rooms) ↗
- Physical Locations:
- British Library DSC - 3405.622000
British Library DSC - BLDSS-3PM
British Library STI - ELD Digital store - Ingest File:
- 3168.xml