Impacts of study design on sample size, participation bias, and outcome measurement: A case study from bicycling research. (December 2019)

• Record Type: Journal Article
• Authors: Branion-Calles, Michael; Winters, Meghan; Nelson, Trisalyn; de Nazelle, Audrey; Panis, Luc Int; Avila-Palencia, Ione; Anaya-Boig, Esther; Rojas-Rueda, David; Dons, Evi; Götschi, Thomas
• Is Part Of: Journal of transport & health. Volume 15(2020)
• Publication Date: 2019-12
• Subjects: Bicycling -- Bias -- Exposure -- Survey participation -- Longitudinal -- Cross-sectional -- Study design; Transportation -- Health aspects -- Periodicals; Transportation -- Periodicals; Public Health -- Periodicals; Noise, Transportation -- Periodicals; Air Pollutants -- Periodicals; 388
• DOI: 10.1016/j.jth.2019.100651
• Languages: English
• ISSNs: 2214-1405

Abstract

Abstract: Introduction: Measuring bicycling behaviour is critical to bicycling research. A common study design question is whether to measure bicycling behaviour once (cross-sectional) or multiple times (longitudinal). The Physical Activity through Sustainable Transport Approaches (PASTA) project is a longitudinal cohort study of over 10, 000 participants from seven European cities over two years. We used PASTA data as a case study to investigate how measuring once or multiple times impacted three factors: a) sample size b) participation bias and c) accuracy of bicycling behaviour estimates. Methods: We compared two scenarios: i) as if only the baseline data were collected (cross-sectional approach) and ii) as if the baseline plus repeat follow-ups were collected (longitudinal approach). We compared each approach in terms of differences in sample size, distribution of sociodemographic characteristics, and bicycling behaviour. In the cross-sectional approach, we measured participants long-term bicycling behaviour by asking for recall of typical weekly habits, while in the longitudinal approach we measured by taking the average of bicycling reported for each 7-day period. Results: Relative to longitudinal, the cross-sectional approach provided a larger sample size and slightly better representation of certain sociodemographic groups, with worse estimates of long-term bicycling behaviour. The longitudinal approach suffered from participation bias, especially the drop-out of more frequent bicyclists. The cross-sectional approach under-estimated the proportion of the population that bicycled, as it captured 'typical' behaviour rather than 7-day recall. The magnitude and directionality of the difference between typical weekly (cross-sectional approach) and the average 7-day recall (longitudinal approach) varied depending on how much bicycling was initially reported. Conclusions: In our case study we found that measuring bicycling once, resulted in a larger sample with better representation of sociodemographic groups, but different estimates of long-term bicycling behaviour. Passive detection of bicycling through mobile apps could be a solution to the identified issues. Highlights: Compared measuring bicycling once versus multiple times in survey. Measuring once results in larger sample, better sociodemographic representation. Measuring multiple times results in substantially different estimates of bicycling. Compared to direct recall, questions about "typical" bicycling habits are biased. Researchers should consider trade-offs between bias and accuracy in design approach.