Data mining fool's gold. (September 2020)

• Record Type: Journal Article
• Authors: Smith, Gary
• Is Part Of: Journal of information technology. Volume 35:Number 3(2020)
• Page Start: 182
• Page End: 194
• Publication Date: 2020-09
• Subjects: Data mining -- data exploration -- knowledge discovery -- artificial intelligence -- machine learning -- HARKing; Information technology -- Periodicals; Information storage and retrieval systems -- Periodicals; Information resources management -- Periodicals; 658.40380285
• DOI: 10.1177/0268396220915600
• Languages: English
• ISSNs: 0268-3962

Abstract

The scientific method is based on the rigorous testing of falsifiable conjectures. Data mining, in contrast, puts data before theory by searching for statistical patterns without being constrained by prespecified hypotheses. Artificial intelligence and machine learning systems, for example, often rely on data-mining algorithms to construct models with little or no human guidance. However, a plethora of patterns are inevitable in large data sets, and computer algorithms have no effective way of assessing whether the patterns they unearth are truly useful or meaningless coincidences. While data mining sometimes discovers useful relationships, the data deluge has caused the number of possible patterns that can be discovered relative to the number that are genuinely useful to grow exponentially—which makes it increasingly likely that what data mining unearths is likely to be fool's gold.