Levenberg–Marquardt multi-classification using hinge loss function. (November 2021)

• Record Type: Journal Article
• Authors: Ozyildirim, Buse Melis; Kiran, Mariam
• Is Part Of: Neural networks. Volume 143(2021)
• Page Start: 564
• Page End: 571
• Publication Date: 2021-11
• Subjects: Neural networks -- Levenberg–Marquardt -- Hinge loss -- Loss functions -- Classification; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.07.010
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Incorporating higher-order optimization functions, such as Levenberg–Marquardt (LM) have revealed better generalizable solutions for deep learning problems. However, these higher-order optimization functions suffer from very large processing time and training complexity especially as training datasets become large, such as in multi-view classification problems, where finding global optima is a very costly problem. To solve this issue, we develop a solution for LM-enabled classification with, to the best of knowledge first-time implementation of hinge loss, for multiview classification. Hinge loss allows the neural network to converge faster and perform better than other loss functions such as logistic or square loss rates. We prove our method by experimenting with various multiclass classification challenges of varying complexity and training data size. The empirical results show the training time and accuracy rates achieved, highlighting how our method outperforms in all cases, especially when training time is limited. Our paper presents important results in the relationship between optimization and loss functions and how these can impact deep learning problems.