Coping with selection effects: a Primer on regression with truncated data. Issue 4 (31st January 2019)

• Record Type: Journal Article
• Authors: Mantz, Adam B
• Is Part Of: Monthly notices of the Royal Astronomical Society. Volume 485:Issue 4(2019)
• Page Start: 4863
• Page End: 4872
• Publication Date: 2019-01-31
• Subjects: methods: data analysis -- methods: statistical; Astronomy -- Periodicals; Periodicals; 520.5
• DOI: 10.1093/mnras/stz320
• Languages: English
• ISSNs: 0035-8711

Abstract

Abstract: The finite sensitivity of instruments or detection methods means that data sets in many areas of astronomy, for example cosmological or exoplanet surveys, are necessarily systematically incomplete. Such data sets, where the population being investigated is of unknown size and only partially represented in the data, are called 'truncated' in the statistical literature. Truncation can be accounted for through a relatively straightforward modification to the model being fitted in many circumstances, provided that the model can be extended to describe the population of undetected sources. Here, I examine the problem of regression using truncated data in general terms, and use a simple example to show the impact of selecting a subset of potential data on the dependent variable, on the independent variable, and on a second dependent variable that is correlated with the variable of interest. Special circumstances in which selection effects are ignorable are noted. I also comment on computational strategies for performing regression with truncated data, as an extension of methods that have become popular for the non-truncated case, and provide some general recommendations.