Self-feeding training method for semi-supervised grammatical error correction. (January 2023)

• Record Type: Journal Article
• Authors: Kwon, Soonchoul; Lee, Gary Geunbae
• Is Part Of: Computer speech & language. Volume 77(2023)
• Publication Date: 2023-01
• Subjects: 68T50; Natural language processing -- Grammatical error correction -- Data augmentation; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2022.101435
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: Grammatical error correction (GEC) has been successful with deep and complex neural machine translation models, but the annotated data to train the model are scarce. We propose a novel self-feeding training method that generates incorrect sentences from freely available correct sentences. The proposed training method can generate appropriate wrong sentences from unlabeled sentences, using a data generation model trained as an autoencoder. It can also add artificial noise to correct sentences to automatically generate incorrect sentences. We show that the GEC models trained with the self-feeding training method are successful without extra annotated data or deeper neural network-based models, achieving F0.5 score of 0.5982 on the CoNLL-2014 Shared Task test data with a transformer model. The results also show that fully unlabeled training is possible for data-scarce domains and languages. Highlights: Self-feeding training method can help training grammatical error correction models. Data generation models trained with unlabeled data can generate realistic data. Denoising artificial noise generates various wrong data pairs for training. This method can be extended to domains and languages without labeled data.