Optimising transparency, reliability and replicability: annotation principles and inter-coder agreement in the quantification of evaluative expressions. Issue 3 (November 2015)

• Record Type: Journal Article
• Authors: Fuoli, Matteo; Hommerberg, Charlotte
• Is Part Of: Corpora. Volume 10:Issue 3(2015)
• Page Start: 315
• Page End: 349
• Publication Date: 2015-11
• Subjects: Appraisal theory -- BP Deepwater Horizon oil spill -- evaluation -- inter-coder agreement -- manual corpus annotation -- reliability -- replicability -- transparency -- trust repair; Corpora (Linguistics) -- Periodicals; 410.188
• DOI: 10.3366/cor.2015.0080
• Languages: English
• ISSNs: 1749-5032

Abstract

Abstract : Manual corpus annotation facilitates exhaustive and detailed corpus-based analyses of evaluation that would not be possible with purely automatic techniques. However, manual annotation is a complex and subjective process. Most studies adopting this approach have paid insufficient attention to the methodological challenges involved in manually annotating evaluation – especially concerning transparency, reliability and replicability. This article illustrates a procedure for annotating evaluative expressions in text that facilitates more transparent, reliable and replicable analyses. The method is demonstrated through a case study analysis ofappraisal (Martin and White, 2005 ) in a small-size specialised corpus of CEO letters published by the British energy company, BP, and four competitors before and after the Deepwater Horizon oil spill of 2010. Drawing onFuoli and Paradis's (2014) model of trust–repair discourse, we examine howattitude andengagement resources are strategically deployed by BP's CEO in the attempt to repair stakeholders' trust after the accident.