Inferactive data analysis. (10th December 2019)

• Record Type: Journal Article
• Authors: Bi, Nan; Markovic, Jelena; Xia, Lucy; Taylor, Jonathan
• Is Part Of: Scandinavian journal of statistics. Volume 47:Number 1(2020:Mar.)
• Page Start: 212
• Page End: 249
• Publication Date: 2019-12-10
• Subjects: confirmatory and exploratory data analysis -- randomization -- selective inference; Statistics -- Periodicals; 310
• DOI: 10.1111/sjos.12425
• Languages: English
• ISSNs: 0303-6898

Abstract

Abstract: We describe inferactive data analysis, so‐named to denote an interactive approach to data analysis with an emphasis on inference after data analysis. Our approach is a compromise between Tukey's exploratory and confirmatory data analysis allowing also for Bayesian data analysis. We see this as a useful step in concrete providing tools (with statistical guarantees) for current data scientists. The basis of inference we use is (a conditional approach to) selective inference, in particular its randomized form. The relevant reference distributions are constructed from what we call a DAG‐DAG—a Data Analysis Generative DAG, and a selective change of variables formula is crucial to any practical implementation of inferactive data analysis via sampling these distributions. We discuss a canonical example of an incomplete cross‐validation test statistic to discriminate between black box models, and a real HIV dataset example to illustrate inference after making multiple queries on data.