On Optimal Correlation-Based Prediction. Issue 4 (2nd October 2022)

• Record Type: Journal Article
• Authors: Bottai, Matteo; Kim, Taeho; Lieberman, Benjamin; Luta, George; Peña, Edsel
• Is Part Of: American statistician. Volume 76:Issue 4(2022)
• Page Start: 313
• Page End: 321
• Publication Date: 2022-10-02
• Subjects: Calibration -- Coefficient of determination -- Concordance correlation coefficient -- Least-squares predictor -- Maximal agreement predictor -- Pearson's correlation; Statistics -- Periodicals; 001.42205
• DOI: 10.1080/00031305.2022.2051604
• Languages: English
• ISSNs: 0003-1305

Abstract

Abstract: This note examines, at the population-level, the approach of obtaining predictors h ˜ ( X ) of a random variable Y, given the joint distribution of ( Y, X ), by maximizing the mapping h ↦ κ ( Y, h ( X ) ) for a given correlation function κ ( ·, · ) . Commencing with Pearson's correlation function, the class of such predictors is uncountably infinite. The least-squares predictor h * is an element of this class obtained by equating the expectations of Y and h ( X ) to be equal and the variances of h ( X ) and E ( Y | X ) to be also equal. On the other hand, replacing the second condition by the equality of the variances of Y and h ( X ), a natural requirement for some calibration problems, the unique predictor h * * that is obtained has the maximum value of Lin's (1989 ) concordance correlation coefficient (CCC) with Y among all predictors. Since the CCC measures the degree of agreement, the new predictor h * * is called the maximal agreement predictor. These predictors are illustrated for three special distributions: the multivariate normal distribution; the exponential distribution, conditional on covariates; and the Dirichlet distribution. The exponential distribution is relevant in survival analysis or in reliability settings, while the Dirichlet distribution is relevant for compositional data.