Model Adequacy Checking for Applying Harmonic Regression to Assessment Quality Control. Issue 1 (8th August 2021)

• Record Type: Journal Article
• Authors: Qian, Jiahe; Li, Shuhong
• Is Part Of: ETS research report series. Issue 1(2021)
• Page Start: 1
• Page End: 26
• Publication Date: 2021-08-08
• Subjects: Sequential data with seasonality -- model selection -- harmonic pair index -- leave‐one‐out cross‐validation -- out‐of‐sample cross‐validation -- rotated jackknife grouping; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; Universities and colleges -- United States -- Entrance examinations; Universities and colleges -- United States -- Graduate work -- Examinations; Graduate Record Examination; Educational tests and measurements; Social sciences; Education -- Research; Education -- Research; Educational tests and measurements; Graduate Record Examination; Social sciences; Universities and colleges -- Entrance examinations; Universities and colleges -- Graduate work -- Examinations; United States; 378
• DOI: 10.1002/ets2.12327
• Languages: English
• ISSNs: 2330-8516

Abstract

Abstract: In recent years, harmonic regression models have been applied to implement quality control for educational assessment data consisting of multiple administrations and displaying seasonality. As with other types of regression models, it is imperative that model adequacy checking and model fit be appropriately conducted. However, there has been no literature on how to perform a comprehensive model adequacy evaluation when applying harmonic regression models to sequential data with seasonality in the educational assessment field. This paper is intended to fill this gap with an illustration of real data from an English language assessment. Two types of cross‐validation, leave‐one‐out and out‐of‐sample, were designed to measure prediction errors and check model validation. Three types of R ‐squared ( R 2, R adj 2, and R pred 2 ) and various residual diagnostics were applied to check model adequacy and model fitting. Abstract : Report Number: RR‐21‐13