"Trust but Verify": The Difficulty of Trusting Autonomous Weapons Systems. Issue 1 (2nd January 2018)

• Record Type: Journal Article
• Authors: Roff, Heather M.; Danks, David
• Is Part Of: Journal of military ethics. Volume 17:Issue 1(2018)
• Page Start: 2
• Page End: 20
• Publication Date: 2018-01-02
• Subjects: Autonomous weapons systems -- trust -- reliability -- training and doctrine -- autonomy; Military ethics -- Periodicals; 172.42
• DOI: 10.1080/15027570.2018.1481907
• Languages: English
• ISSNs: 1502-7570

Abstract

ABSTRACT: Autonomous weapons systems (AWSs) pose many challenges in complex battlefield environments. Previous discussions of them have largely focused on technological or policy issues. In contrast, we focus here on the challenge of trust in an AWS. One type of human trust depends only on judgments about the predictability or reliability of the trustee, and so are suitable for all manner of artifacts. However, AWSs that are worthy of the descriptor "autonomous" will not exhibit the required strong predictability in the complex, changing contexts of war. Instead, warfighters need to develop deeper, interpersonal trust that is grounded in understanding the values, beliefs, and dispositions of the AWS. Current acquisition, training, and deployment processes preclude the development of such trust, and so there are currently no routes for a warfighter to develop trust in an AWS. We thus survey three possible changes to current practices in order to facilitate the type of deep trust that is required for appropriate, ethical use of AWSs.