Kernel density estimation with linked boundary conditions. Issue 3 (18th June 2020)

• Record Type: Journal Article
• Authors: Colbrook, Matthew J.; Botev, Zdravko I.; Kuritz, Karsten; MacNamara, Shev
• Is Part Of: Studies in applied mathematics. Volume 145:Issue 3(2020)
• Page Start: 357
• Page End: 396
• Publication Date: 2020-06-18
• Subjects: boundary bias -- biological cell cycle -- density estimation -- diffusion -- linked boundary conditions -- unified transform; Mathematics -- Periodicals; Mathématiques -- Périodiques; Mathématique; Mathématique appliquée; Ressource Internet (Descripteur de forme); Périodique électronique (Descripteur de forme); 519
• DOI: 10.1111/sapm.12322
• Languages: English
• ISSNs: 0022-2526

Abstract

Abstract: Kernel density estimation on a finite interval poses an outstanding challenge because of the well‐recognized bias at the boundaries of the interval. Motivated by an application in cancer research, we consider a boundary constraint linking the values of the unknown target density function at the boundaries. We provide a kernel density estimator (KDE) that successfully incorporates this linked boundary condition, leading to a non‐self‐adjoint diffusion process and expansions in nonseparable generalized eigenfunctions. The solution is rigorously analyzed through an integral representation given by the unified transform (or Fokas method). The new KDE possesses many desirable properties, such as consistency, asymptotically negligible bias at the boundaries, and an increased rate of approximation, as measured by the AMISE. We apply our method to the motivating example in biology and provide numerical experiments with synthetic data, including comparisons with state‐of‐the‐art KDEs (which currently cannot handle linked boundary constraints). Results suggest that the new method is fast and accurate. Furthermore, we demonstrate how to build statistical estimators of the boundary conditions satisfied by the target function without a priori knowledge. Our analysis can also be extended to more general boundary conditions that may be encountered in applications.