The Right Accounting of Wrongs: Examining Temporal Changes to Human Rights Monitoring and Reporting. Issue 1 (7th January 2023)

• Record Type: Journal Article
• Authors: Arnon, Daniel; Haschke, Peter; Park, Baekkwan
• Is Part Of: British journal of political science. Volume 53:Issue 1(2023)
• Page Start: 163
• Page End: 182
• Publication Date: 2023-01-07
• Subjects: human rights measurement -- bias -- machine learning -- natural language processing; Political science -- Periodicals; 320.05
• DOI: 10.1017/S0007123421000661
• Languages: English
• ISSNs: 0007-1234

Abstract

Abstract: Scholars contend that the reason for stasis in human rights measures is a biased measurement process, rather than stagnating human rights practices. We argue that bias may be introduced as part of the compilation of the human rights reports that serve as the foundation of human rights measures. An additional source of potential bias may be human coders, who translate human rights reports into human rights scores. We first test for biases via a machine-learning approach using natural language processing and find substantial evidence of bias in human rights scores. We then present findings of an experiment on the coders of human rights reports to assess whether potential changes in the coding procedures or interpretation of coding rules affect scores over time. We find no evidence of coder bias and conclude that human rights measures have changed over time and that bias is introduced as part of monitoring and reporting.