Mutual equidistant-scattering criterion: A new index for crisp clustering. (15th August 2019)

• Record Type: Journal Article
• Authors: Flexa, Caio; Santos, Reginaldo; Gomes, Walisson; Sales, Claudomiro; Costa, João C.W.A.
• Is Part Of: Expert systems with applications. Volume 128(2019)
• Page Start: 225
• Page End: 245
• Publication Date: 2019-08-15
• Subjects: Cluster validity index -- Mutual equidistant-scattering -- Number of clusters -- Crisp clustering; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2019.03.027
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A new non-parametric internal validity index is proposed for crisp clustering. The index is based on within-cluster mutual equidistant-scattering. It is proved effective on both (4) real-world and (26) synthetic benchmark data sets. Seven other indexes are employed in comparisons to detect the number of clusters. The Friedman's test corroborates that better results are yielded with this proposal. Abstract: Clustering algorithms usually assume that the number K of clusters is known, although there is often no prior knowledge about the underlying set. Consequently, the significance of the defined groups needs to be validated. Cluster validity indexes are commonly used to perform the validation of clustering results. However, most of them are considered to be dependent on the number of data objects and often tend to ignore small and low-density groups. Furthermore, suboptimal clustering solutions are frequently selected when the clusters are in a certain degree of overlapping or low separation. Thus, we propose a new non-parametric internal validity index based on within-cluster mutual equidistant-scattering for crisp clustering. Eight different validity indexes were analysed to detect the number of clusters in a data set. Experiments on both synthetic and real-world data show the effectiveness and reliability of our approach to evaluate the hyperparameter K .