Group-attention Based Neural Machine Translation. Issue 2 (March 2020)

• Record Type: Journal Article
• Authors: Hu, Renyu; Xu, Hao; Xiao, Yang; Wu, Chenjun; Jia, Haiyang
• Is Part Of: IOP conference series. Volume 782:Issue 2(2020)
• Publication Date: 2020-03
• Subjects: Materials science -- Periodicals; 620.1105
• DOI: 10.1088/1757-899X/782/2/022080
• Languages: English
• ISSNs: 1757-8981

Abstract

Abstract: Machine translation is a classic problem in natural language process (NLP). Recent years, the encoder and decoder through an attention mechanism has become a trend. Google proposed a new simple network architecture, the Transformer using attention mechanisms only rather than CNN or RNN in 2017. However, it may lose some important information (e.g., grammatical component, etc) when using attention mechanism for whole sentence.We propose a new brand model based on transformer using Group attention layers and group position embedding. It absorbs the features of Group-CNN combines the algorithm in computer vision (CV) and NLP. The model not only pays more attention to the ingredients (e.g., subject, predicate and adverbial, etc), but also enhances the connection of phrase. It outperforms SofA Transformer in using more syntactic information.