A crowdsourced system for user studies in information extraction. (2017)

• Record Type: Journal Article
• Authors: Khojasteh-Ghamari, Zohreh
• Is Part Of: International journal of knowledge engineering and soft data paradigms. Volume 6:Number 1(2017)
• Page Start: 44
• Page End: 51
• Publication Date: 2017
• Subjects: crowdsourcing -- information extraction -- data mining; Soft computing -- Periodicals; Statistics -- Periodicals; Information science -- Periodicals; 003.05
• Languages: English
• ISSNs: 1755-3210

Abstract

In this paper, from an entity linking (EL) system, we take a set of tweets, where some subsequence of words is annotated with possible meaning/entities and these entities are linked with several Wikipedia pages. We propose a model using crowdsourcing to disambiguate and decide about the accurate Wikipedia page that must be linked with a definite word/spot. We discuss about importance of crowdsourcing and compare different crowdsourcing systems and at the end, introduce crowdflower. We discuss about the crowdflower features in particular. Finally, we analyse output reports of the crowdflower and present a novel approach to select the reliable results. In summary, our observations show that reliable results have a confidence rate over 0.5.