How Accurate Are Survey Responses on Social Media and Politics?. (3rd April 2019)

• Record Type: Journal Article
• Authors: Guess, Andrew; Munger, Kevin; Nagler, Jonathan; Tucker, Joshua
• Is Part Of: Political communication. Volume 36:Number 2(2019)
• Page Start: 241
• Page End: 258
• Publication Date: 2019-04-03
• Subjects: social media -- media effects -- survey measurement; Communication -- Political aspects -- Periodicals; Communication in politics -- Periodicals; 306.205
• DOI: 10.1080/10584609.2018.1504840
• Languages: English
• ISSNs: 1058-4609

Abstract

Abstract : How accurate are survey-based measures of social media use, in particular about political topics? We answer this question by linking original survey data collected during the U.S. 2016 election campaign with respondents' observed social media activity. We use supervised machine learning to classify whether these Twitter and Facebook account data are content related to politics. We then benchmark our survey measures on frequency of posting about politics and the number of political figures followed. We find that, on average, our self-reported survey measures tend to correlate with observed social media activity. At the same time, we also find a worrying amount of individual-level discrepancy and problems related to extreme outliers. Our recommendations are twofold. The first is for survey questions about social media use to provide respondents with options covering a wider range of activity, especially in the long tail. The second is for survey questions to include specific content and anchors defining what it means for a post to be "about politics."