A Pseudo-Likelihood Approach to Linear Regression With Partially Shuffled Data. Issue 4 (2nd October 2021)

• Record Type: Journal Article
• Authors: Slawski, Martin; Diao, Guoqing; Ben-David, Emanuel
• Is Part Of: Journal of computational and graphical statistics. Volume 30:Issue 4(2021)
• Page Start: 991
• Page End: 1003
• Publication Date: 2021-10-02
• Subjects: Broken sample problem -- Expectation-maximization algorithm -- Mixture models -- Pseudo-likelihood -- Record linkage; Mathematical statistics -- Data processing -- Periodicals; Mathematical statistics -- Graphic methods -- Periodicals; 519.50285
• DOI: 10.1080/10618600.2020.1870482
• Languages: English
• ISSNs: 1061-8600

Abstract

Abstract: Recently, there has been significant interest in linear regression in the situation where predictors and responses are not observed in matching pairs corresponding to the same statistical unit as a consequence of separate data collection and uncertainty in data integration. Mismatched pairs can considerably impact the model fit and disrupt the estimation of regression parameters. In this article, we present a method to adjust for such mismatches under "partial shuffling" in which a sufficiently large fraction of (predictors, response)-pairs are observed in their correct correspondence. The proposed approach is based on a pseudo-likelihood in which each term takes the form of a two-component mixture density. expectation-maximization schemes are proposed for optimization, which (i) scale favorably in the number of samples, and (ii) achieve excellent statistical performance relative to an oracle that has access to the correct pairings as certified by simulations and case studies. In particular, the proposed approach can tolerate considerably larger fraction of mismatches than existing approaches, and enables estimation of the noise level as well as the fraction of mismatches. Inference for the resulting estimator (standard errors, confidence intervals) can be based on established theory for composite likelihood estimation. Along the way, we also propose a statistical test for the presence of mismatches and establish its consistency under suitable conditions. Supplemental files for this article are available online.