Fit for Purpose in Action: Design, Implementation, and Evaluation of the National Internet Flu Survey. (9th March 2020)

• Record Type: Journal Article
• Authors: Dever, Jill A; Amaya, Ashley; Srivastav, Anup; Lu, Peng-Jun; Roycroft, Jessica; Stanley, Marshica; Stringer, M Christopher; Bostwick, Michael G; Greby, Stacie M; Santibanez, Tammy A; Williams, Walter W
• Is Part Of: Journal of Survey Statistics and Methodology. Volume 9:Number 3(2021)
• Page Start: 449
• Page End: 476
• Publication Date: 2020-03-09
• Subjects: Computer-assisted web interview (CAWI) -- External validity -- Fit for purpose -- Influenza vaccination -- Probability-based internet panel -- Quality metrics -- Survey design; Surveys -- Methodology -- Periodicals; Surveys -- Evaluation -- Periodicals; Sampling (Statistics) -- Periodicals; 001.433
• DOI: 10.1093/jssam/smz050
• Languages: English
• ISSNs: 2325-0984

Abstract

Abstract: Researchers strive to design and implement high-quality surveys to maximize the utility of the data collected. The definitions of quality and usefulness, however, vary from survey to survey and depend on the analytic needs. Survey teams must evaluate the trade-offs of various decisions, such as when results are needed and their required level of precision, in addition to practical constraints like budget, before finalizing the design. Characteristics within the concept of fit for purpose (FfP) can provide the framework for considering the trade-offs. Furthermore, this tool can enable an evaluation of quality for the resulting estimates. Implementation of a FfP framework in this context, however, is not straightforward. In this article, we provide the reader with a glimpse of a FfP framework in action for obtaining estimates on early season influenza vaccination coverage estimates and on knowledge, attitudes, behaviors, and barriers related to influenza and influenza prevention among civilian noninstitutionalized adults aged 18 years and older in the United States. The result is the National Internet Flu Survey (NIFS), an annual, two-week internet survey sponsored by the US Centers for Disease Control and Prevention. In addition to critical design decisions, we use the established NIFS FfP framework to discuss the quality of the NIFS in meeting the intended objectives. We highlight aspects that work well and other survey traits requiring further evaluation. Differences found in comparing the NIFS to the National Flu Survey, the National Health Interview Survey, and Behavioral Risk Factor Surveillance System are discussed via their respective FfP characteristics. The findings presented here highlight the importance of the FfP framework for designing surveys, defining data quality, and providing a set a metrics used to advertise the intended use of the survey data and results.