Generation of Coherent Multi-Sentence Texts with a Coherence Mechanism. (March 2023)

• Record Type: Journal Article
• Authors: Zhao, Qingjuan; Niu, Jianwei; Liu, Xuefeng; He, Wenbo; Tang, Shaojie
• Is Part Of: Computer speech & language. Volume 78(2023)
• Publication Date: 2023-03
• Subjects: Text generation -- Multiple sentences summarization -- Attention mechanism -- Coherence mechanism -- Abstractive summarization; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2022.101457
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: Automatic generation of long texts containing multiple sentences has many applications in the field of Natural Language Processing (NLP) including question answering, machine translation, and paraphrase generation, etc. However, in terms of readability, the long texts generated by machines are not comparable to those organized by human beings. Through statistics, we observed that human-organized texts generally have a special property: one or more of the words (particularly nouns and pronouns) appeared in one sentence will reappear in the next one in the same or a different form. This repetition of words in consecutive sentences can greatly improve the readability. Based on this observation, we propose CMST, a deep neural network model for generating C oherent M ulti-S entence T exts. CMST explicitly incorporates a training strategy of coherence mechanism to evaluate the repetition of words in consecutive sentences. We evaluate the performance of the CMST on the CNN/Daily Mail dataset. The experimental results show that, compared with the baseline models, CMST not only improves the readability of the generated texts, but achieves higher METEOR and ROUGE values. Highlights: The reappearance of words in adjacent sentences could make the text read coherently. The coherence level is defined and incorporated into the objective function. The results show that the generated texts achieve higher evaluation scores.