How Linkage Error Affects Hidden Markov Model Estimates: A Sensitivity Analysis. (29th May 2019)

• Record Type: Journal Article
• Authors: Pankowska, Paulina; Bakker, Bart F M; Oberski, Daniel L; Pavlopoulos, Dimitris
• Is Part Of: Journal of Survey Statistics and Methodology. Volume 8:Number 3(2020)
• Page Start: 483
• Page End: 512
• Publication Date: 2019-05-29
• Subjects: Classification error -- Hidden Markov model (HMM) -- Linkage error -- Measurement error -- Misclassification -- Record linkage; Surveys -- Methodology -- Periodicals; Surveys -- Evaluation -- Periodicals; Sampling (Statistics) -- Periodicals; 001.433
• DOI: 10.1093/jssam/smz011
• Languages: English
• ISSNs: 2325-0984

Abstract

Abstract: Hidden Markov models (HMMs) are increasingly used to estimate and correct for classification error in categorical, longitudinal data, without the need for a "gold standard, " error-free data source. To accomplish this, HMMs require multiple observations over time on a single indicator and assume that the errors in these indicators are conditionally independent. Unfortunately, this "local independence" assumption is often unrealistic, untestable, and a source of serious bias. Linking independent data sources can solve this problem by making the local independence assumption plausible across sources, while potentially allowing for local dependence within sources. However, record linkage introduces a new problem: the records may be erroneously linked or incorrectly not linked. In this paper, we investigate the effects of linkage error on HMM estimates of transitions between employment contract types. Our data come from linking a labor force survey to administrative employer records; this linkage yields two indicators per time point that are plausibly conditionally independent. Our results indicate that both false-negative and false-positive linkage error turn out to be problematic primarily if the error is large and highly correlated with the dependent variable. Moreover, under certain conditions, false-positive linkage error (mislinkage) in fact acts as another source of misclassification that the HMM can absorb into its error-rate estimates, leaving the latent transition estimates unbiased. In these cases, measurement error modeling already accounts for linkage error. Our results also indicate where these conditions break down and more complex methods would be needed.