A confidence-prioritisation approach for learning noisy data. (30th December 2014)

• Record Type: Journal Article
• Authors: Gustafson, Nathaniel; Giraud-Carrier, Christophe
• Is Part Of: International journal of data analysis techniques and strategies. Volume 6:Number 4(2014)
• Page Start: 307
• Page End: 326
• Publication Date: 2014-12-30
• Subjects: data mining -- confidence prioritisation -- diel signal prediction -- applications -- noisy data -- learning -- environmental data; Electronic data processing -- Periodicals; Database searching -- Periodicals; 005
• Languages: English
• ISSNs: 1755-8050

Abstract

In a number of real-world applications, there is a range of noise associated with individual data points. Some points are extracted under relatively clear and defined conditions, while others may be affected by a variety of known or unknown confounding factors, which may decrease those points' validity. These points may or may not remain useful for training, depending on how much uncertainty they contain. We submit that in situations where some variability exists in the clarity or confidence associated with individual data points, an approach that takes this confidence into account during the training phase is beneficial. We propose a methodological framework for assigning confidence to individual data records and augmenting training with that information. We test the methodology on two separate datasets, a simulated dataset and a streamflow diel signals dataset. Results indicate that applying and utilising confidence in training improves performance.