List Experiments with Measurement Error. (20th May 2019)

• Record Type: Journal Article
• Authors: Blair, Graeme; Chou, Winston; Imai, Kosuke
• Is Part Of: Political analysis. Volume 27:Number 4(2019)
• Page Start: 455
• Page End: 480
• Publication Date: 2019-05-20
• Subjects: auxiliary information, -- indirect questioning, -- item count technique, -- misspecification test, -- sensitive survey questions, -- unmatched count technique; Political science -- Methodology -- Periodicals; Electronic journals; 320.011
• DOI: 10.1017/pan.2018.56
• Languages: English
• ISSNs: 1047-1987

Abstract

Abstract : Measurement error threatens the validity of survey research, especially when studying sensitive questions. Although list experiments can help discourage deliberate misreporting, they may also suffer from nonstrategic measurement error due to flawed implementation and respondents' inattention. Such error runs against the assumptions of the standard maximum likelihood regression (MLreg ) estimator for list experiments and can result in misleading inferences, especially when the underlying sensitive trait is rare. We address this problem by providing new tools for diagnosing and mitigating measurement error in list experiments. First, we demonstrate that the nonlinear least squares regression (NLSreg ) estimator proposed in Imai (2011 ) is robust to nonstrategic measurement error. Second, we offer a general model misspecification test to gauge the divergence of theMLreg andNLSreg estimates. Third, we show how to model measurement error directly, proposing new estimators that preserve the statistical efficiency ofMLreg while improving robustness. Last, we revisit empirical studies shown to exhibit nonstrategic measurement error, and demonstrate that our tools readily diagnose and mitigate the bias. We conclude this article with a number of practical recommendations for applied researchers. The proposed methods are implemented through an open-source software package.