Discovering phonetic inventories with crosslingual automatic speech recognition. (July 2022)

• Record Type: Journal Article
• Authors: Żelasko, Piotr; Feng, Siyuan; Moro Velázquez, Laureano; Abavisani, Ali; Bhati, Saurabhchand; Scharenborg, Odette; Hasegawa-Johnson, Mark; Dehak, Najim
• Is Part Of: Computer speech & language. Volume 74(2022)
• Publication Date: 2022-07
• Subjects: Phone inventory -- ASR -- Speech recognition -- Multilingual -- Crosslingual -- Zero-shot -- Phone recognition -- Speech representation; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2022.101358
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: The high cost of data acquisition makes Automatic Speech Recognition (ASR) model training problematic for most existing languages, including languages that do not even have a written script, or for which the phone inventories remain unknown. Past works explored multilingual training, transfer learning, as well as zero-shot learning in order to build ASR systems for these low-resource languages. While it has been shown that the pooling of resources from multiple languages is helpful, we have not yet seen a successful application of an ASR model to a language unseen during training. A crucial step in the adaptation of ASR from seen to unseen languages is the creation of the phone inventory of the unseen language. The ultimate goal of our work is to build the phone inventory of a language unseen during training in an unsupervised way without any knowledge about the language. In this paper, we (1) investigate the influence of different factors (i.e., model architecture, phonotactic model, type of speech representation) on phone recognition in an unknown language; (2) provide an analysis of which phones transfer well across languages and which do not in order to understand the limitations of and areas for further improvement for automatic phone inventory creation; and (3) present different methods to build a phone inventory of an unseen language in an unsupervised way. To that end, we conducted mono-, multi-, and crosslingual experiments on a set of 13 phonetically diverse languages and several in-depth analyses. We found a number of universal phone tokens (IPA symbols) that are well-recognized cross-linguistically. Through a detailed analysis of results, we conclude that unique sounds, similar sounds, and tone languages remain a major challenge for phonetic inventory discovery. Graphical abstract: Highlights: We perform phonetic inventory discovery and present a way of approaching it with ASR. We train mono-, multi-, and crosslingual hybrid and E2E ASR on 13 diverse languages. Not knowing the target language phonotactics a priori is a limitation. We found universal phone tokens: IPA symbols that are well-recognized cross-linguistically. Unique sounds, similar sounds, and tone languages remain a challenge.