Chinese Text Summarization Generation Algorithm based on ERNIE-GEN with Few-shot Learning. Issue 1 (July 2021)

• Record Type: Journal Article
• Authors: Zhu, Yongqing; Zhao, Peng; Mu, Xiaodong; Bai, Kun; You, Xuanang
• Is Part Of: Journal of physics. Volume 1961:Issue 1(2021)
• Publication Date: 2021-07
• Subjects: Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/1961/1/012051
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: In view of the problems of exposure bias and insufficient annotation data in the existing Chinese short text summary algorithms, a new pre-training short text summary algorithm named ERNIE-GEN-CTS based on ERNIE-GEN is proposed. This paper uses pre-training language model to realize the double attention of word level and word level for short text, and fills the sub block with semantic information to optimize the exposure bias problem. After the pre-training model of large-scale Chinese corpus, the results are improved in LCSTS data set after the task tuning of the downstream summary, and the validity and stability of the model are verified.