Deep reinforcement and transfer learning for abstractive text summarization: A review. (January 2022)

• Record Type: Journal Article
• Authors: Alomari, Ayham; Idris, Norisma; Sabri, Aznul Qalid Md; Alsmadi, Izzat
• Is Part Of: Computer speech & language. Volume 71(2022)
• Publication Date: 2022-01
• Subjects: Abstractive summarization -- Sequence-to-sequence -- Reinforcement learning -- Pre-trained models; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2021.101276
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: ATS field overview: We provide a full review of the ATS field, providing. A taxonomy with different categories. A brief history of models' evolution. Evaluation measurements review. Datasets comparisons. ATS and MT research fields comparison and relationship. Models comprehensive review: We collect abundant resources on the main topics of this study and provide a comprehensive review of SOTA research work: Starting from Deep neural sequence-to-sequence models, then RL approaches, and finally TL architectures, including PTLMs. Challenges: We analyze previous and current challenges that faced and are facing researchers in the focused fields and the proposed solutions. Comparisons: We provide different kinds of comparisons of the investigated models from different perspectives: theoretically, practically, and models' evaluation results. Then the best models are highlighted. Future trends: We suggest and discuss possible future research trends. Abstract: Automatic Text Summarization (ATS) is an important area in Natural Language Processing (NLP) with the goal of shortening a long text into a more compact version by conveying the most important points in a readable form. ATS applications continue to evolve and utilize effective approaches that are being evaluated and implemented by researchers. State-of-the-Art (SotA) technologies that demonstrate cutting-edge performance and accuracy in abstractive ATS are deep neural sequence-to-sequence models, Reinforcement Learning (RL) approaches, and Transfer Learning (TL) approaches, including Pre-Trained Language Models (PTLMs). The graph-based Transformer architecture and PTLMs have influenced tremendous advances in NLP applications. Additionally, the incorporation of recent mechanisms, such as the knowledge-enhanced mechanism, significantly enhanced the results. This study provides a comprehensive review of recent research advances in the area of abstractive text summarization for works spanning the past six years. Past and present problems are described, as well as their proposed solutions. In addition, abstractive ATS datasets and evaluation measurements are also highlighted. The paper concludes by comparing the best models and discussing future research directions.