Assessing the impact of minor modifications on the interior structure of GRU: GRU1 and GRU2. (13th December 2021)

• Record Type: Journal Article
• Authors: Yigit, Gulsum; Amasyali, Mehmet Fatih
• Other Names: Wright Steven A. guestEditor.; Solak Serdar guestEditor.; Kilimci Zeynep Hilal guestEditor.; Eken Süleyman guestEditor.; Fernandes Steven guestEditor.; Zhang Yu‐Dong guestEditor.; Tavares João Manuel R.S. guestEditor.
• Is Part Of: Concurrency and computation. Volume 34:Number 20(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-12-13
• Subjects: curriculum learning -- gated recurrent units -- recurrent neural networks -- Seq2seq -- short‐term dependency; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.6775
• Languages: English
• ISSNs: 1532-0626

Abstract

Abstract: In this study, two GRU variants named GRU1 and GRU2 are proposed by employing simple changes to the internal structure of the standard GRU, which is one of the popular RNN variants. Comparative experiments are conducted on four problems: language modeling, question answering, addition task, and sentiment analysis. Moreover, in the addition task, curriculum learning and anti‐curriculum learning strategies, which extend the training data having examples from easy to hard or from hard to easy, are comparatively evaluated. Accordingly, the GRU1 and GRU2 variants outperformed the standard GRU. In addition, the curriculum learning approach, in which the training data is expanded from easy to difficult, improves the performance considerably.