Clustering as physically inspired energy minimization. (February 2019)

• Record Type: Journal Article
• Authors: Yang, Huiguang; Ahuja, Narendra
• Is Part Of: Pattern recognition. Volume 86(2019:Feb.)
• Page Start: 265
• Page End: 280
• Publication Date: 2019-02
• Subjects: Unsupervised/hierarchical clustering -- Energy minimization -- Statistical physics -- Integer programming -- Unary/data energy -- Ising model -- Local density -- Connected component analysis -- Image segmentation -- Normalized-cut; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2018.09.008
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We more completely map the energy model of statistical physics onto clustering problem, and our method can be totally unsupervised. We make a perfect analogy with the energy model used in vision and borrow the methods from vision field to clustering under this mapping. We propose a data point local density estimation method which can account for the datasets with arbitrary shapes and topologies. We point out that the energy model of spectral clustering methods (such as Normalized-cut[22] ) is incomplete compared with our energy model. Abstract: We formulate the task of density based clustering as energy minimization, using both binary/pairwise energy term and unary/data energy term (the latter was largely ignored in previous clustering methods). Binary energy is defined in terms of inhomogeneity in local point density. While most previous methods use binary/pairwise energy only, the unary/data energy can represent the natural tendency of a given point belonging to a given cluster, which is also crucial for the clustering. Since our energy is expressed as the sum of a unary (data) term and a binary (pairwise or smoothness) term, we can thus make a perfect analogy with the energy model used in vision and borrow everything (such as the optimization algorithms) from vision field to clustering under this mapping. This correspondence provides an entirely new view point in handling the clustering problem, and in fact many mature methods and algorithms are already provided in the vision field and can be adopted by the clustering field readily. During our energy optimization, a sequence of energy minima are found to recursively partition the points, and thus find a hierarchical embedding of clusters that are increasingly homogeneous in density. Disjoint clusters with the same density are identified separately. Our clustering method is totally unsupervised (which is superior to most existing methods, as those listed below). It does not need any user input parameters (say number of segments, any bandwidth parameter, cutoff distance/scale, etc.), except one can specify the homogeneity criterion — the degree of acceptable fluctuation in density within a cluster (which is target-related), or let it be specified automatically in a hierarchical way. We conduct experiments on both synthetic datasets and real-image tasks. Experimental results on synthetic datasets show that our method is able to handle clusters of different shapes, sizes and densities. We present the performance of our approach using the commonly used energy optimization algorithms from vision such as ICM, LBP, Graph-cut, Mean field theory algorithm, as well as the integer programming algorithm. We also contrast our performance with several other commonly used clustering algorithms, such as k -means, fussy c -means, DBSCAN, as well as a recent state-of-the-art clustering method as reported in [40]. Our experiments on real-image tasks further validate the performance of the method. In addition, we show that the family of commonly used spectral, graph clustering algorithms (such as Normalized-cut) uses only the binary energy term while ignoring the unary energy term; therefore, their energy model is incomplete compared with ours.