Understanding the merits of winning data competition solutions for varied sets of objectives. (29th December 2020)

• Record Type: Journal Article
• Authors: Lu, Lu; Anderson‐Cook, Christine M.; Zhang, Miaolu
• Other Names: Morris Max D. guestEditor.
• Is Part Of: Statistical analysis and data mining. Volume 14:Number 6(2021)
• Page Start: 556
• Page End: 574
• Publication Date: 2020-12-29
• Subjects: comparison of algorithms -- data competitions -- graphical summaries -- multi‐objective optimization -- objective versus subjective -- Pareto fronts; Data mining -- Statistical methods -- Periodicals; 006.312
• DOI: 10.1002/sam.11494
• Languages: English
• ISSNs: 1932-1864

Abstract

Abstract: Data competitions provide an efficient cost‐effective way to obtain diverse solutions for challenging problems across a wide variety of applications. The competition leaderboard, by necessity, must combine multiple objectives into a single scoring formula to determine winners and allocate prize money. However, after the competition concludes, the host may wish to choose a best solution for a particular scenario that focuses on only a subset of all the competition objectives. Through the use of Pareto fronts and graphical summaries, we describe how top solutions for a specific scenario can be identified and compared. The strategy uses intentional tie‐handling, thresholds to eliminate undesirable solutions and Pareto fronts to identify objectively superior solutions for a subset of objectives. Then the strengths and weaknesses of different alternatives can be compared to find the ideal solution for the problem. The methods are illustrated with a real Topcoder data competition hosted by Los Alamos National Laboratory that used 16 different objectives to evaluate the quality of solutions for urban radiation search.