A probabilistic metric for comparing metaheuristic optimization algorithms. (January 2018)

• Record Type: Journal Article
• Authors: Gomes, Wellison J.S.; Beck, André T.; Lopez, Rafael H.; Miguel, Leandro F.F.
• Is Part Of: Structural safety. Volume 70(2018)
• Page Start: 59
• Page End: 70
• Publication Date: 2018-01
• Subjects: Optimization algorithms -- Metaheuristic -- Evolutionary algorithms -- Performance metric -- Population interference; Structural stability -- Periodicals; Safety factor in engineering -- Periodicals; Reliability (Engineering) -- Periodicals; Constructions -- Stabilité -- Périodiques; Coefficient de sécurité en ingénierie -- Périodiques; Fiabilité -- Périodiques; 620.86
• DOI: 10.1016/j.strusafe.2017.10.006
• Languages: English
• ISSNs: 0167-4730

Abstract

Highlights: Metaheuristic optimization algorithm runs are based on random numbers. Several runs required for comparing performance of algorithms. Single probabilistic metric proposed for ranking metaheuristic optimization algorithms. Metric yields probability that given algorithm is better than alternative, in a single run. Metric also quantifies how much better algorithm is. Abstract: The evolution of metaheuristic optimization algorithms towards identification of a global minimum is based on random numbers, making each run unique. Comparing the performance of different algorithms hence requires several runs, and some statistical metric of the results. Mean, standard deviation, best and worst values metrics have been used with this purpose. In this paper, a single probabilistic metric is proposed for comparing metaheuristic optimization algorithms. It is based on the idea of population interference, and yields the probability that a given algorithm produces a smaller (global?) minimum than an alternative algorithm, in a single run. Three benchmark example problems and four optimization algorithms are employed to demonstrate that the proposed metric is better than usual statistics such as mean, standard deviation, best and worst values obtained over several runs. The proposed metric actually quantifies how much better a given algorithm is, in comparison to an alternative algorithm. Statements about the superiority of an algorithm can also be made in consideration of the number of algorithm runs and the number of objective function evaluations allowed in each run.