A General Hybrid Clustering Technique. Issue 3 (3rd July 2019)

• Record Type: Journal Article
• Authors: Amiri, Saeid; Clarke, Bertrand S.; Clarke, Jennifer L.; Koepke, Hoyt
• Is Part Of: Journal of computational and graphical statistics. Volume 28:Issue 3(2019)
• Page Start: 540
• Page End: 551
• Publication Date: 2019-07-03
• Subjects: Consistency -- K-mean -- Lifetime -- Outlier -- Single linkage -- t-SNE; Mathematical statistics -- Data processing -- Periodicals; Mathematical statistics -- Graphic methods -- Periodicals; 519.50285
• DOI: 10.1080/10618600.2018.1546593
• Languages: English
• ISSNs: 1061-8600

Abstract

Abstract: Here, we propose a clustering technique for general clustering problems including those that have nonconvex clusters. For a given desired number of clusters K, we use three stages to find clusters. The first stage uses a hybrid clustering technique to produce a series of clusterings of various sizes (randomly selected). The key step in this stage is to find a K -means clustering using K ℓ clusters where K ℓ ≫ K and then join these small clusters by using single linkage clustering. The second stage stabilizes the result of stage one by reclustering via the "membership matrix" under Hamming distance to generate a dendrogram. The third stage is to cut the dendrogram to get K * clusters where K * ≥ K and then prune back to K to give a final clustering. A variant on our technique also gives a reasonable estimate for KT, the true number of clusters. We provide arguments to justify the steps in the stages of our methods and we provide examples involving simulated and published data to compare our technique with other techniques. An R library, GHC, implementing our method is available from Github.