Automated selection of changepoints using empirical P-values and trimming. Issue 4 (29th October 2022)

• Record Type: Journal Article
• Authors: Quinn, Matthew; Chung, Arlene; Glass, Kimberly
• Is Part Of: JAMIA open. Volume 5:Issue 4(2022)
• Publication Date: 2022-10-29
• Subjects: mobile health -- changepoints -- time series -- Monte Carlo method -- regression; Medical informatics -- Periodicals; 610.285
• DOI: 10.1093/jamiaopen/ooac090
• Languages: English
• ISSNs: 2574-2531

Abstract

Abstract: Objectives: One challenge that arises when analyzing mobile health (mHealth) data is that updates to the proprietary algorithms that process these data can change apparent patterns. Since the timings of these updates are not publicized, an analytic approach is necessary to determine whether changes in mHealth data are due to lifestyle behaviors or algorithmic updates. Existing methods for identifying changepoints do not consider multiple types of changepoints, may require prespecifying the number of changepoints, and often involve nonintuitive parameters. We propose a novel approach, Automated Selection of Changepoints using Empirical P -values and Trimming (ASCEPT), to select an optimal set of changepoints in mHealth data. Materials and Methods: ASCEPT involves 2 stages: (1) identification of a statistically significant set of changepoints from sequential iterations of a changepoint detection algorithm; and (2) trimming changepoints within linear and seasonal trends. ASCEPT is available at https://github.com/matthewquinn1/changepointSelect . Results: We demonstrate ASCEPT's utility using real-world mHealth data collected through the Precision VISSTA study. We also demonstrate that ASCEPT outperforms a comparable method, circular binary segmentation, and illustrate the impact when adjusting for changepoints in downstream analysis. Discussion: ASCEPT offers a practical approach for identifying changepoints in mHealth data that result from algorithmic updates. ASCEPT's only required parameters are a significance level and goodness-of-fit threshold, offering a more intuitive option compared to other approaches. Conclusion: ASCEPT provides an intuitive and useful way to identify which changepoints in mHealth data are likely the result of updates to the underlying algorithms that process the data. Lay Summary: Mobile health (mHealth) has taken on an increasingly important role in medicine and public health. However, how mHealth data are reported to an end user can change based on updates to the underlying proprietary algorithms that process these data. The times at which these changes occur represent potential "changepoints." Effectively using mHealth data for health applications requires correctly identifying when such algorithmic changes occur and distinguishing them from changes in mHealth data that are due to lifestyle behaviors. We present Automated Selection of Changepoints using Empirical P -values and Trimming (ASCEPT) as a method to identify changepoints in mHealth data. ASCEPT correctly identifies algorithmic changepoints both in the context of simulated data and real Fitbit data collected as part of the Precision VISSTA study. We also find that ASCEPT outperforms a comparable approach, circular binary segmentation, and that this difference in performance has a clear impact when adjusting for the identified changepoints. ASCEPT offers an intuitive and useful approach for identifying changepoints in mHealth data before performing any downstream analysis.