Using ANOVA/random-effects variance estimates to compute a two-sample U-statistic of order (1, 1) estimate of variance. (2nd January 2016)

• Record Type: Journal Article
• Authors: Tcheuko, Lucas; Gallas, Brandon; Samuelson, Frank
• Is Part Of: Journal of statistical theory and practice. Volume 10:Number 1(2016)
• Page Start: 87
• Page End: 99
• Publication Date: 2016-01-02
• Subjects: AUC -- ROC -- variance components -- rank statistic -- ANOVA; 62G05; 519.505
• DOI: 10.1080/15598608.2015.1077759
• Languages: English
• ISSNs: 1559-8608

Abstract

ABSTRACT: The classical empirical, area under the receiver operating characteristic (ROC) curve (AUC) is a two-sample U -statistic of order (1, 1). Its variance can be written out as a sum of three tractable covariances. It is then possible to consider each of these covariances as U -statistics themselves and follow the U -statistics formalism to derive their unbiased estimates. Over the years, alternative methods have been proposed to estimate the variance of AUC. For example, Delong et al. have proposed a straightforward estimate that has attractive asymptotic properties. At small sample sizes, however, the DeLong method will be biased. In the early stage of investigation, researchers don't always have enough data; therefore, those asymptotic variance estimates such as DeLong's can be unreliable. In this article we propose a two-way random effects analysis of variance (ANOVA) method to compute an unbiased variance estimate of a two-sample U -statistic of order (1, 1) in general, and of the AUC in particular. We prove that this variance estimate is equal to the fully U -statistic result. In the particular case of the AUC variance estimate we compare our result to DeLong's AUC variance estimate. We extend the result to obtain an unbiased estimate of variance for a linear combination of possibly correlated AUCs. A natural consequence of this extension is the estimate of variance of the difference of the areas under two correlated ROC curves. This difference is of interest in many diagnostic studies, namely, the comparison of two different modalities used to diagnose the same disease.