Non-monotonic convergence of online learning algorithms for perceptrons with noisy teacher. (June 2018)

• Record Type: Journal Article
• Authors: Ikeda, Kazushi; Honda, Arata; Hanzawa, Hiroaki; Miyoshi, Seiji
• Is Part Of: Neural networks. Volume 102(2018)
• Page Start: 21
• Page End: 26
• Publication Date: 2018-06
• Subjects: Learning curve -- Perceptron -- Online learning -- Statistical mechanics -- Asymptotic analysis; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2018.02.009
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Learning curves of simple perceptron were derived here. The learning curve of the perceptron learning with noisy teacher was shown to be non-monotonic, which has never appeared even though the learning curves have been analyzed for half a century. In this paper, we showed how this phenomenon occurs by analyzing the asymptotic property of the perceptron learning using a method in systems science, that is, calculating the eigenvalues of the system matrix and the corresponding eigenvectors. We also analyzed the AdaTron learning and the Hebbian learning in the same way and found that the learning curve of the AdaTron learning is non-monotonic whereas that of the Hebbian learning is monotonic.