Optimal clustering of frequency data with application to disease risk categorization. (3rd August 2022)

• Record Type: Journal Article
• Authors: Aprahamian, Hrayer; El-Amine, Hadi
• Is Part Of: IISE transactions. Volume 54:Number 8(2022)
• Page Start: 728
• Page End: 740
• Publication Date: 2022-08-03
• Subjects: Cluster analysis -- set partitioning problem -- constrained shortest path -- hierarchical clustering -- disease risk categorization; Industrial engineering -- Periodicals; Systems engineering -- Periodicals; Industrial engineering; Systems engineering; Electronic journals; Periodicals; 670.285
• DOI: Https://www.tandfonline.com/doi/10.1080/24725854.2021.1973158
• Languages: English
• ISSNs: 2472-5854

Abstract

Abstract: We provide a clustering procedure for a special type of dataset, known as frequency data, which counts the frequency of a certain binary outcome. An interpretation of the data as a discrete distribution enables us to extract statistical information, which we embed within an optimization-based framework. Our analysis of the resulting combinatorial optimization problem allows us to reformulate it as a more tractable network flow problem. This, in turn, enables the construction of exact algorithms that converge to the optimal solution in quadratic time. In addition, to be able to handle large-scale datasets, we provide two hierarchical heuristic algorithms that run in linearithmic time. Our moment-based method results in clustering solutions that are shown to perform well for a family of applications. We illustrate the benefits of our findings through a case study on HIV risk categorization within the context of large-scale screening through group testing. Our results on CDC data show that the proposed clustering framework consistently outperforms other popular clustering methods.