F-statistic for model validation over experimental regions using least squares response surfaces. (2017)

• Record Type: Journal Article
• Authors: Storm, Scott M.; Hill, Raymond R.; Pignatiello, Joseph J.; White, Edward D.; Vining, G. Geoffrey
• Is Part Of: International journal of experimental design and process optimisation. Volume 5:Number 3(2017)
• Page Start: 133
• Page End: 150
• Publication Date: 2017
• Subjects: experimental design -- F-statistic -- linear regression -- mean square error -- response surface -- simulation -- stochastic model validation; Experimental design -- Periodicals; Process control -- Statistical methods -- Periodicals; 620.0072
• Languages: English
• ISSNs: 2040-2252

Abstract

When a simulation operates over an array of input settings, it is critical that its validity across all settings is considered. This paper proposes an F-statistic, based on a ratio of mean square errors, to assess model validity. System data is fit to an Ordinary Least Squares response surface and the mean square errors between both system and model data and the fitted surface are calculated. The hypothesis under test is that system and model data that possess the same distributional properties will produce similar measures of disagreement with regard to the fitted surface. If the F-statistic indicates the mean square errors are statistically different, it is determined that the system and model data do not share distributional properties and the model is assessed as invalid. A notional example demonstrates the methodology and considerations to the method's confidence with respect to the model sample size is discussed.