Machine learning from crowds: A systematic review of its applications. (16th October 2018)

• Record Type: Journal Article
• Authors: G. Rodrigo, Enrique; Aledo, Juan A.; Gámez, José A.
• Is Part Of: Wiley interdisciplinary reviews. Volume 9:Number 2(2019)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2018-10-16
• Subjects: applications -- crowdsourcing -- labeler quality -- machine learning -- multiple annotation; Data mining -- Periodicals; 006.31205
• DOI: 10.1002/widm.1288
• Languages: English
• ISSNs: 1942-4787

Abstract

Abstract : Crowdsourcing opens the door to solving a wide variety of problems that previously were unfeasible in the field of machine learning, allowing us to obtain relatively low cost labeled data in a small amount of time. However, due to the uncertain quality of labelers, the data to deal with are sometimes unreliable, forcing practitioners to collect information redundantly, which poses new challenges in the field. Despite these difficulties, many applications of machine learning using crowdsourced data have recently been published that achieved state of the art results in relevant problems. We have analyzed these applications following a systematic methodology, classifying them into different fields of study, highlighting several of their characteristics and showing the recent interest in the use of crowdsourcing for machine learning. We also identify several exciting research lines based on the problems that remain unsolved to foster future research in this field. This article is categorized under: Technologies > Machine Learning Application Areas > Science and Technology Fundamental Concepts of Data and Knowledge > Key Design Issues in Data Mining Abstract : Thanks to crowdsourcing platforms, several problems that seemed impossible to tackle from a machine learning perspective, can now be solved. To discover how crowdsourced data is used as well as what are the main problems derived from its use, we analyze many applications in several fields of interest.