Assessing text mining algorithm outcomes. Issue 2 (2nd July 2020)

• Record Type: Journal Article
• Authors: Ashton, Triss; Evangelopoulos, Nicholas; Paswan, Audhesh; Prybutok, Victor R.; Pavur, Robert
• Is Part Of: Journal of Business Analytics. Volume 3:Issue 2(2020)
• Page Start: 107
• Page End: 121
• Publication Date: 2020-07-02
• Subjects: Text mining -- algorithm testing -- model development -- latent semantic analysis -- latent Dirichlet allocation; Business intelligence -- Periodicals; Management -- Statistical methods -- Periodicals; Decision making -- Statistical methods -- Periodicals; 658.403
• DOI: 10.1080/2573234X.2020.1785342
• Languages: English
• ISSNs: 2573-234X

Abstract

ABSTRACT: There is a surge in the development of decision-oriented analysis tools intended to extract actionable information from text. These tools integrate various text-mining methods that were performance tested in a manner that was often biased toward the new system. Those tests primarily utilised descriptive measurement criteria and test datasets that are inconsistent with most business corpora. We propose and test a user-oriented judgment approach that allows testing under controlled customer-oriented corpora and generates effect size measures. To illustrate the approach, customer relations data was analysed by latent semantic analysis and latent Dirichlet analysis with results evaluated by prospective business analysts. Reporting includes comparisons of results with published literature. While the research centres on the context-region text-mining systems, literature comparisons include word-embedding methods. The analysis concludes that none of the systems reviewed possess a repeatable statistical advantage over the others. Instead, distribution attributes, algorithm configuration, and the evaluation task drive results.