Behavioral Experiments With Social Algorithms: An Information Theoretic Approach to Input–Output Conversions. Issue 4 (2nd October 2019)

• Record Type: Journal Article
• Authors: Hilbert, Martin; Liu, Billy; Luu, Jonathan; Fishbein, Joel
• Is Part Of: Communication methods and measures. Volume 13:Issue 4(2019)
• Page Start: 267
• Page End: 286
• Publication Date: 2019-10-02
• Subjects: Communication -- Methodology -- Periodicals; Communication -- Research -- Periodicals; Communication -- Study and teaching -- Periodicals; 302.2072
• DOI: 10.1080/19312458.2019.1620712
• Languages: English
• ISSNs: 1931-2458

Abstract

ABSTRACT: While traditional computer-mediated communication happened through transparent, passive, and neutral channels, today's communication channels are obscure, proactive, and distorted. Social algorithms, guided by a socio-technological codependency, often bias communication, usually in pursuit of some third-party goal of commercial or political nature. We propose a method to derive several summary measures to tests for transformational accuracy when transforming input into an output. Since dynamical flexibility of social algorithms prevents anticipating their behavior, we study these black boxes as if we study human behavior, through controlled experiments. We conceptualize them as noisy communication channels and evaluate their throughput with the same information theoretic measures engineers had originally used to minimize communicative distortion (i.e., mutual information). We use repeated experiments to reverse-engineer algorithmic behavior and test for its statistical significance. We apply the method to three artificial intelligence algorithms: a neural net from IBM's Watson, and to the recommender engines of YouTube and Twitter.