Seeking relationships in big data: a Bayesian perspective. Issue 2 (2nd April 2016)

• Record Type: Journal Article
• Authors: Singpurwalla, Nozer D.
• Is Part Of: International journal of management science and engineering management. Volume 11:Issue 2(2016)
• Page Start: 116
• Page End: 121
• Publication Date: 2016-04-02
• Subjects: Association -- Bayesian -- big data -- correlation -- dependence; C11 -- C35 -- C46 -- C53 -- C83; Management science -- Periodicals; Engineering -- Management -- Periodicals; Engineering -- Management; Management science; Periodicals; 658.005
• DOI: 10.1080/17509653.2016.1140405
• Languages: English
• ISSNs: 1750-9661

Abstract

Abstract : The real purpose of collecting big data is to identify causality in the hope that this will facilitate credible predictivity. But the search for causality can trap one into infinite regress, and thus one takes refuge in seeking associations between variables in data sets. Regrettably, the mere knowledge of associations does not enable predictivity. Associations need to be embedded within the framework of the probability calculus to make coherent predictions. This is so because associations are a feature of probability models, and hence they do not exist outside the framework of a model. Measures of association, like correlation, regression, and mutual information merely refute a preconceived model. Estimated measures of associations do not lead to a probability model; a model is the product of pure thought. This paper discusses these and other fundamentals that are germane to seeking associations in particular, and machine learning in general.