An investigation on the factors affecting machine learning classifications in gamma-ray astronomy. Issue 4 (20th January 2020)

• Record Type: Journal Article
• Authors: Luo, Shengda; Leung, Alex P; Hui, C Y; Li, K L
• Is Part Of: Monthly notices of the Royal Astronomical Society. Volume 492:Issue 4(2020)
• Page Start: 5377
• Page End: 5390
• Publication Date: 2020-01-20
• Subjects: methods: statistical -- pulsars: general -- gamma-rays: stars; Astronomy -- Periodicals; Periodicals; 520.5
• DOI: 10.1093/mnras/staa166
• Languages: English
• ISSNs: 0035-8711

Abstract

ABSTRACT: We have investigated a number of factors that can have significant impacts on the classification performance of gamma-ray sources detected by Fermi Large Area Telescope (LAT) with machine learning techniques. We show that a framework of automatic feature selection can construct a simple model with a small set of features that yields better performance over previous results. Secondly, because of the small sample size of the training/test sets of certain classes in gamma-ray, nested re-sampling and cross-validations are suggested for quantifying the statistical fluctuations of the quoted accuracy. We have also constructed a test set by cross-matching the identified active galactic nuclei (AGNs) and the pulsars (PSRs) in the Fermi- LAT 8-yr point source catalogue (4FGL) with those unidentified sources in the previous 3rd Fermi -LAT Source Catalog (3FGL). Using this cross-matched set, we show that some features used for building classification model with the identified source can suffer from the problem of covariate shift, which can be a result of various observational effects. This can possibly hamper the actual performance when one applies such model in classifying unidentified sources. Using our framework, both AGN/PSR and young pulsar (YNG)/millisecond pulsar (MSP) classifiers are automatically updated with the new features and the enlarged training samples in 4FGL catalogue incorporated. Using a two-layer model with these updated classifiers, we have selected 20 promising MSP candidates with confidence scores $\gt 98{{\ \rm per\ cent}}$ from the unidentified sources in 4FGL catalogue that can provide inputs for a multiwavelength identification campaign.