Comparison of rank aggregation methods based on inherent ability. (14th December 2017)

• Record Type: Journal Article
• Authors: Xiao, Yu; Deng, Ye; Wu, Jun; Deng, Hong‐Zhong; Lu, Xin
• Is Part Of: Naval research logistics. Volume 64:Number 7(2017)
• Page Start: 556
• Page End: 565
• Publication Date: 2017-12-14
• Subjects: comparison -- decision analysis -- experimental data generation method -- inherent ability -- ranking -- rank aggregation; Logistics, Naval -- Periodicals; Supplies and stores -- Periodicals; 359.07
• DOI: 10.1002/nav.21771
• Languages: English
• ISSNs: 0894-069X

Abstract

Abstract: Ranking is a common task for selecting and evaluating alternatives. In the past few decades, combining rankings results from various sources into a consensus ranking has become an increasingly active research topic. In this study, we focus on the evaluation of rank aggregation methods. We first develop an experimental data generation method, which can provide ground truth ranking for alternatives based on their "inherent ability." This experimental data generation method can generate the required individual synthetic rankings with adjustable accuracy and length. We propose characterizing the effectiveness of rank aggregation methods by calculating the Kendall tau distance between the aggregated ranking and the ground truth ranking. We then compare four classical rank aggregation methods and present some useful findings on the relative performances of the four methods. The results reveal that both the accuracy and length of individual rankings have a remarkable effect on the comparison results between rank aggregation methods. Our methods and results may be helpful to both researchers and decision‐makers.