Evaluation theory and its application to military assessments. (October 2019)

• Record Type: Journal Article
• Authors: Keith, Andrew; Ahner, Darryl; Curtis, Nicole
• Is Part Of: Journal of defense modeling and simulation. Volume 16:Number 4(2019:Oct.)
• Page Start: 305
• Page End: 322
• Publication Date: 2019-10
• Subjects: Operation assessment -- evaluation theory -- uncertainty -- risk -- ambiguity; Military art and science -- Computer simulation -- Periodicals; 355.0011305
• DOI: 10.1177/1548512919834670
• Languages: English
• ISSNs: 1548-5129

Abstract

Evaluation theory provides a rigorous foundation for the practice of military operation assessment. Government and industry assessors have used evaluation theory to improve the effectiveness of assessment across a wide range of fields. This article focuses on the relationship between evaluation theory and military assessment. We briefly survey the major evaluation approaches with a focus on connecting the theoretical models to practical security-related applications. These evaluation approaches include expertise-oriented, program-oriented, decision-oriented, and participant-oriented models. Within the overarching framework of these approaches, we consider alternative monitoring and evaluation designs in detail, including descriptive designs (case study, cross-sectional, time-series), quasi-experimental designs (interrupted time-series, comparison group, case study), and experimental designs (posttest-only, pre-post). Then, we discuss quantitative and qualitative methods for analyzing and reporting uncertainty with respect to each design alternative, with an emphasis on mixed-method approaches. Throughout the review, we make the relationship between evaluation theory and operation assessment practice explicit through examples, and we suggest more detailed references where appropriate.