An unethical optimization principle. Issue 7 (1st July 2020)

• Record Type: Journal Article
• Authors: Beale, Nicholas; Battey, Heather; Davison, Anthony C.; MacKay, Robert S.
• Is Part Of: Royal Society open science. Volume 7:Issue 7(2020)
• Publication Date: 2020-07-01
• Subjects: AI ethics -- artificial intelligence -- economics -- extreme value theory -- financial regulation; Science -- Periodicals; 500
• DOI: 10.1098/rsos.200462
• Languages: English
• ISSNs: 2054-5703

Abstract

Abstract : If an artificial intelligence aims to maximize risk-adjusted return, then under mild conditions it is disproportionately likely to pick an unethical strategy unless the objective function allows sufficiently for this risk. Even if the proportion η of available unethical strategies is small, the probability p U of picking an unethical strategy can become large; indeed, unless returns are fat-tailed p U tends to unity as the strategy space becomes large. We define an unethical odds ratio, Υ (capital upsilon), that allows us to calculate p U from η, and we derive a simple formula for the limit of Υ as the strategy space becomes large. We discuss the estimation of Υ and p U in finite cases and how to deal with infinite strategy spaces. We show how the principle can be used to help detect unethical strategies and to estimate η . Finally we sketch some policy implications of this work.