Collaborative Annotation for Reliable Natural Language Processing : Technical and Sociological Aspects /: Technical and Sociological Aspects. (2016)

• Record Type: Book
• Further Information: Note: Karën Fort.
• Other Names: Fort, Karën
• Contents: Preface ix List of Acronyms xi Introduction xiii Chapter 1. Annotating Collaboratively 1 1.1. The annotation process (re)visited 1 1.1.1. Building consensus 1 1.1.2. Existing methodologies 3 1.1.3. Preparatory work 7 1.1.4. Pre-campaign 13 1.1.5. Annotation 17 1.1.6. Finalization 21 1.2. Annotation complexity 24 1.2.1. Example overview 25 1.2.2. What to annotate? 28 1.2.3. How to annotate? 30 1.2.4. The weight of the context 36 1.2.5. Visualization 38 1.2.6. Elementary annotation tasks 40 1.3. Annotation tools 43 1.3.1. To be or not to be an annotation tool 43 1.3.2. Much more than prototypes 46 1.3.3. Addressing the new annotation challenges 49 1.3.4. The impossible dream tool 54 1.4. Evaluating the annotation quality 55 1.4.1. What is annotation quality? 55 1.4.2. Understanding the basics 56 1.4.3. Beyond kappas 63 1.4.4. Giving meaning to the metrics 67 1.5. Conclusion 75 Chapter 2. Crowdsourcing Annotation 77 2.1. What is crowdsourcing and why should we be interested in it? 77 2.1.1. A moving target 77 2.1.2. A massive success 80 2.2. Deconstructing the myths 81 2.2.1. Crowdsourcing is a recent phenomenon 81 2.2.2. Crowdsourcing involves a crowd (of non-experts) 83 2.2.3. “Crowdsourcing involves (a crowd of) non-experts” 87 2.3. Playing with a purpose 93 2.3.1. Using the players’ innate capabilities and world knowledge 94 2.3.2. Using the players’ school knowledge 96 2.3.3. Using the players’ learning capacities 97 2.4. Acknowledging crowdsourcing specifics 101 2.4.1. Motivating the participants 101 2.4.2. Producing quality data 107 2.5. Ethical issues 109 2.5.1. Game ethics 109 2.5.2. What’s wrong with Amazon Mechanical Turk? 111 2.5.3. A charter to rule them all 113 Conclusion 115 Appendix 117 Glossary 141 Bibliography 143 Index 163
• Publisher Details: London : John Wiley & Sons
• Publication Date: 2016
• Extent: 1 online resource (vii, 164 pages)
• Subjects: 006.4; COMPUTERS / General; Natural language processing (Computer science); Natural language processing (Computer science); Electronic books
• Languages: English
• ISBNs: 1119307643; 9781119307648; 1119307651; 9781119307655; 9781119306696; 1119306698
• Related ISBNs: 1848219040; 9781848219045
• Notes: Note: Includes bibliographical references and index.; Note: Print version record.
• Access Rights: Legal Deposit; Only available on premises controlled by the deposit library and to one user at any one time; The Legal Deposit Libraries (Non-Print Works) Regulations (UK).
• Access Usage: Restricted: Printing from this resource is governed by The Legal Deposit Libraries (Non-Print Works) Regulations (UK) and UK copyright law currently in force.