Statistical inference in massive data sets. (5th July 2012)

• Record Type: Journal Article
• Authors: Li, Runze; Lin, Dennis K.J.; Li, Bing
• Is Part Of: Applied stochastic models in business and industry. Volume 29:Number 5(2013:Sep./Oct.)
• Page Start: 399
• Page End: 409
• Publication Date: 2012-07-05
• Subjects: Stochastic analysis -- Periodicals; Stochastic processes -- Periodicals; Business mathematics -- Periodicals; Finance -- Mathematical models -- Periodicals; Industrial management -- Mathematical models -- Periodicals; 338.00151923
• DOI: 10.1002/asmb.1927
• Languages: English
• ISSNs: 1524-1904

Abstract

<abstract abstract-type="main" id="asmb1927-abs-0001"> <title> <x xml:space="preserve">Abstract</x> </title> <p id="asmb1927-para-0001">Analysis of massive data sets is challenging owing to limitations of computer primary memory. In this paper, we propose an approach to estimate population parameters from a massive data set. The proposed approach significantly reduces the required amount of primary memory, and the resulting estimate will be as efficient if the entire data set was analyzed simultaneously. Asymptotic properties of the resulting estimate are studied, and the asymptotic normality of the resulting estimator is established. The standard error formula for the resulting estimate is proposed and empirically tested; thus, statistical inference for parameters of interest can be performed. The effectiveness of the proposed approach is illustrated using simulation studies and an Internet traffic data example. Copyright © 2012 John Wiley &amp; Sons, Ltd.</p> </abstract>