Identification of related languages from spoken data: Moving from off-line to on-line scenario. (July 2021)

• Record Type: Journal Article
• Authors: Cerva, Petr; Mateju, Lukas; Zdansky, Jindrich; Safarik, Radek; Nouza, Jan
• Is Part Of: Computer speech & language. Volume 68(2021)
• Publication Date: 2021-07
• Subjects: Spoken language identification -- Deep neural networks -- Weighted finite-state transducers -- On-line processing -- Slavic languages; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2020.101180
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Real-world spoken data streams, e.g., from TV or radio, often contain utterances in several related languages. A novel approach is presented allowing for online identification of the spoken language in this type of multilingual streams. Our method utilizes a decoder and a language classifier fed by multilingual bottleneck features. These features, as well as the classifier, make use of neural network architectures that allow for learning long-term dependencies. The entire online recognition scheme operates with an average latency of around 2.5 s. Abstract: The accelerating flow of information we encounter around the world today makes many companies deploy speech recognition systems that, to an ever-growing extent, process data on-line rather than off-line. These systems, e.g., for real-time 24/7 broadcast transcription, often work with input-stream data containing utterances in more than one language. This multilingual data can correctly be transcribed in real-time only if the language used is identified with just a small latency for each input frame. For this purpose, a novel approach to on-line spoken language identification is proposed in this work. Its development is documented within a series of consecutive experiments starting in the off-line mode for 11 Slavic languages, going through artificially prepared multilingual data for the on-line scenario, and ending with real bilingual TV programs containing utterances in mutually similar Czech and Slovak. The resulting scheme that we propose operates frame-by-frame; it takes in a multilingual stream of speech frames and outputs a stream of the corresponding language labels. It utilizes a weighted finite-state transducer as a decoder, which smooths the output from a language classifier fed by multilingual and augmented bottleneck features. An essential factor from the accuracy point of view is that these features, as well as the classifier itself, are based on deep neural network architectures that allow the modeling of long-term time dependencies. The obtained results show that our scheme allows us to determine the language spoken in real-world bilingual TV shows with an average latency of around 2.5 seconds and with an increase in word error rate by a mere 2.9% over the reference 18.1% value yielded by using manually prepared language labels.