Modified online Newton step based on elementwise multiplication. (27th February 2020)

• Record Type: Journal Article
• Authors: Singh, Charanjeet; Sharma, Anuj
• Is Part Of: Computational intelligence. Volume 36:Number 3(2020)
• Page Start: 1010
• Page End: 1025
• Publication Date: 2020-02-27
• Subjects: machine learning -- online learning -- online Newton step -- elementwise multiplication -- second‐order online learning; Artificial intelligence -- Periodicals; Computational linguistics -- Periodicals; 006.3
• DOI: 10.1111/coin.12298
• Languages: English
• ISSNs: 0824-7935

Abstract

Abstract: The second‐order method using a Newton step is a suitable technique in online learning to guarantee a regret bound. The large data are a challenge in the Newton method to store second‐order matrices such as the hessian. In this article, we have proposed a modified online Newton step that stores first‐ and second‐order matrices of dimension m (classes) by d (features). We have used elementwise arithmetic operations to maintain the size of matrices. The modified second‐order matrix size results in faster computations. Also, the mistake rate is on par with respect to popular methods in the literature. The experimental outcome indicates that proposed method could be helpful to handle large multiclass datasets on common desktop machines using second‐order method as the Newton step.