RepSum: A general abstractive summarization framework with dynamic word embedding representation correction. (May 2023)

• Record Type: Journal Article
• Authors: Feng, Jianzhou; Long, Jing; Han, Chunlong; Ren, Zhongcan; Wang, Qin
• Is Part Of: Computer speech & language. Volume 80(2023)
• Publication Date: 2023-05
• Subjects: 68T50 -- 91F20; Abstractive text summarization -- Dynamic word embedding -- Representation correction algorithm -- Word frequency features elimination; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2023.101491
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: Abstractive summarization is flexible and allows the model to generate new words and phrases. However, the familiar words are more likely to be selected as abstract candidate words in the process of abstractive summarization, causing the generated abstract to diverge from the reference. In our consideration, this is caused by representation degeneration of the pre-trained word embedding. Therefore, this paper proposes a general abstractive summarization framework with dynamic word embedding representation correction (RepSum ). The representation correction algorithm identifies the dimension most relevant to word frequency and eliminates the word frequency features. Then the distribution of word embeddings will be more even. As a result, the words will be selected as candidate words without frequency bias to improve the quality of the abstract. The experimental results illustrate that RepSum performs better than the benchmark model in summary quality, demonstrating our method's effectiveness. Graphical abstract: Highlights: An embedding representation correction algorithm based on word frequency feature elimination is proposed. The algorithm identifies the dimension most relevant to word frequency and eliminates the word frequency features. Then the distribution of word embeddings will be more even and the words will be selected as candidate words without frequency bias.