A guide to statistical analysis for single criteria heuristics evaluation. (23rd January 2008)

• Record Type: Journal Article
• Authors: Adenso-Diaz, Belarmino; Corral, Norberto
• Is Part Of: International journal of innovative computing and applications. Volume 1:Number 2(2007)
• Page Start: 94
• Page End: 107
• Publication Date: 2008-01-23
• Subjects: heuristics analysis -- statistical tests -- algorithmics -- single criteria heuristics; Evolutionary computation -- Periodicals; Neural networks (Computer science) -- Periodicals; Genetic programming (Computer science) -- Periodicals; Biologically-inspired computing -- Periodicals; Swarm intelligence -- Periodicals; Quantum computers -- Periodicals; 006.3
• Languages: English
• ISSNs: 1751-648X

Abstract

The success obtained by metaheuristic techniques in resolving combinatory problems of a real character has led to a veritable explosion in the number of studies in this field. From a scientific perspective, these new approaches must demonstrate improved performance, normally with respect to prior proposals. This involves the carrying out of a series of computational experiments and statistical tests demonstrating their validity. However, in a large number of cases and for diverse causes (a lack of data relative to the benchmark heuristic or due to not knowing the most appropriate statistical tests), some authors fail to present conclusive statistical tests that may be qualified as possessing a minimum level of rigour according to the scientific method. This paper analyses the different tests that may be considered to verify significant differences between the performance of different single criteria heuristics, as well as including a number of application examples, and a discussion about the problem of robustness.