Finding Singular Features. Issue 3 (3rd July 2017)

• Record Type: Journal Article
• Authors: Genovese, Christopher; Perone-Pacifico, Marco; Verdinelli, Isabella; Wasserman, Larry
• Is Part Of: Journal of computational and graphical statistics. Volume 26:Issue 3(2017)
• Page Start: 598
• Page End: 609
• Publication Date: 2017-07-03
• Subjects: Clustering -- Density estimation -- Manifolds -- Ridges; Mathematical statistics -- Data processing -- Periodicals; Mathematical statistics -- Graphic methods -- Periodicals; 519.50285
• DOI: 10.1080/10618600.2016.1260472
• Languages: English
• ISSNs: 1061-8600

Abstract

ABSTRACT: We present a method for finding high density, low-dimensional structures in noisy point clouds. These structures are sets with zero Lebesgue measure with respect to the D -dimensional ambient space and belong to a d < D -dimensional space. We call them "singular features." Hunting for singular features corresponds to finding unexpected or unknown structures hidden in point clouds belonging to R D . Our method outputs well-defined sets of dimensions d < D . Unlike spectral clustering, the method works well in the presence of noise. We show how to find singular features by first finding ridges in the estimated density, followed by a filtering step based on the eigenvalues of the Hessian of the density. The code for plotting all the figures, with the corresponding plots, and the data files used in the article, are in the folder SupplementaryDocument.zip that can be find at the http://www.stat.cmu.edu/larry/singular .