Prescribing Structure for Validation Arguments: Elemental, Structural, and Ecological Validity. Issue 1 (2nd January 2019)

• Record Type: Journal Article
• Authors: Jacobson, Erik; Svetina, Dubravka
• Is Part Of: Applied measurement in education. Volume 32:Issue 1(2019)
• Page Start: 43
• Page End: 59
• Publication Date: 2019-01-02
• Subjects: Educational tests and measurements -- United States -- Periodicals; Educational tests and measurements -- Periodicals; Education -- Standards -- Periodicals; 371.26
• DOI: 10.1080/08957347.2018.1544137
• Languages: English
• ISSNs: 0895-7347

Abstract

ABSTRACT: Contingent argument-based approaches to validity require a unique argument for each use, in contrast to more prescriptive approaches that identify the common kinds of validity evidence researchers should consider for every use. In this article, we evaluate our use of an approach that is both prescriptive and argument-based to develop a validity argument for a novel instrument.The elemental aspect concerns the items; the structural aspect concerns how the items function together; and the ecological aspect concerns how the resulting instrument is related to the context of use. Elemental, structural, and ecological aspects of validity are considered independently. We illustrate this approach by describing our experience constructing a validity argument for a novel, integrated knowledge and motivation instrument for use in teacher education. For each aspect, we illustrate the approach by discussing theory-based assumptions, articulating inferences based on these assumptions, and describing how we collected and assessed empirical evidence to support the inferences. We conclude with a retrospective analysis resulting in lessons learned for others who may consider a prescriptive, argument-based approach to validity.