Uses of artificial and composite treatments in experimental methods: Reconsidering the problem of validity and its implications for stratification research. (February 2020)

• Record Type: Journal Article
• Authors: Chen, Jacqueline Chen; Tam, Tony
• Is Part Of: Research in social stratification and mobility. Volume 65(2020)
• Publication Date: 2020-02
• Subjects: Experimental validity -- Artificial treatment -- Composite treatment -- Inequality preference -- China; Social mobility -- Periodicals; Occupational mobility -- Periodicals; Social status -- Periodicals; Social conflict -- Periodicals; Social classes -- Periodicals; Mobilité sociale -- Périodiques; Mobilité professionnelle -- Périodiques; Statut social -- Périodiques; Conflits sociaux -- Périodiques; Classes sociales -- Périodiques; 305.05
• DOI: 10.1016/j.rssm.2019.100443
• Languages: English
• ISSNs: 0276-5624

Abstract

Highlights: Common criticisms of experiments for a lack of generalizability and realism are misplaced. Artificial and composite treatments as experimental control. Empirically compare three measures of inequality preference of over 62, 000 urban Chinese using two experimental treatments (strict egalitarianism and acceptable merit-based pay differential) and a standard attitudinal item (income differentials as motivator). Perceived fairness of achieved living standard is positively associated with (a) tolerance of merit-based pay differential and, surprisingly, (b) strict generalized egalitarian preference, but not with (c) belief in income differentials as motivator. Abstract: This paper promotes the incorporation of the expanding uses of experimental methods in stratification research and beyond. Social science experiments are usually criticized for their lack of external validity, especially their limited generalizability to real-life settings. We offer a critique of this concern by reaffirming the classical insight of experimental methodology that external validity should not be a key concern and focusing on some under-appreciated scientific payoffs to experimental control. We illustrate the opportunities for productive use of artificial and composite treatments in tandem to obtain three measures of inequality preference: generalized egalitarianism (derived from an incentivized behavioral game), acceptable merit-based economic differential (from a vignette experiment), and motivational value of income differentials (from a standard attitudinal item). This approach enables us to examine their roles in determining the perceived fairness of living standard given one's effort—an outcome of much interest in subjective stratification research. The empirical results are based on a mobile phone survey of over 62, 000 urban respondents, covering more than 330 of the largest cities in China. Using fixed-effects to control for locational unobservables, we show that perceived fairness is positively associated with tolerance of merit-based pay differential and, surprisingly, strict generalized egalitarian preference, but not with belief in income differentials as motivator for effort. Since the seemingly equivalent measures have very different predictive relationships with an individual's perceived fairness, the results offer a cautionary tale of invalid interpretations that overlook the subtle nonequivalence among valid measurement strategies with differing degrees of realism.