A smoothed Q‐learning algorithm for estimating optimal dynamic treatment regimes. (26th December 2018)

• Record Type: Journal Article
• Authors: Fan, Yanqin; He, Ming; Su, Liangjun; Zhou, Xiao‐Hua
• Is Part Of: Scandinavian journal of statistics. Volume 46:Number 2(2019:Jun.)
• Page Start: 446
• Page End: 469
• Publication Date: 2018-12-26
• Subjects: asymptotic normality -- exceptional law -- optimal smoothing parameter -- sequential randomization -- Wald‐type inference; Statistics -- Periodicals; 310
• DOI: 10.1111/sjos.12359
• Languages: English
• ISSNs: 0303-6898

Abstract

Abstract: In this paper, we propose a smoothed Q ‐learning algorithm for estimating optimal dynamic treatment regimes. In contrast to the Q ‐learning algorithm in which nonregular inference is involved, we show that, under assumptions adopted in this paper, the proposed smoothed Q ‐learning estimator is asymptotically normally distributed even when the Q ‐learning estimator is not and its asymptotic variance can be consistently estimated. As a result, inference based on the smoothed Q ‐learning estimator is standard. We derive the optimal smoothing parameter and propose a data‐driven method for estimating it. The finite sample properties of the smoothed Q ‐learning estimator are studied and compared with several existing estimators including the Q ‐learning estimator via an extensive simulation study. We illustrate the new method by analyzing data from the Clinical Antipsychotic Trials of Intervention Effectiveness–Alzheimer's Disease (CATIE‐AD) study.