A hierarchical linguistic information-based model of English prosody: L2 data analysis and implications for computer-assisted language learning. (September 2018)

• Record Type: Journal Article
• Authors: Su, Chao-yu; Tseng, Chiu-yu; Jang, Jyh-Shing Roger; Visceglia, Tanya
• Is Part Of: Computer speech & language. Volume 51(2018)
• Page Start: 44
• Page End: 67
• Publication Date: 2018-09
• Subjects: Prosody -- Discourse structure -- Information structure -- L2 english -- Resynthesis -- CALL -- linguistic -- Continuous speech; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2018.03.001
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: The present study pinpoints prosodic differences between native (L1) and non-native Taiwan (L2) English using hierarchical linguistic information. The hierarchical linguistic information used includes hierarchical discourse association (DS) and information structure (IS). Distinct F0 and duration patterns found in Taiwan L2 English account for Taiwan accent in terms of less sensitivity to both DS and IS. The proposed L1 model is used to simulate native-like expressive prosody that is capable of generating corrective feedback for L2 learners. The corrected feedback helps increase intelligibility and comprehensibility of L2 speech in terms of fluency and expressiveness. Abstract: The paper presents a prosody model of native English (L1) continuous speech as corrective prosodic feedback for non-native learners. The model incorporates both hierarchical discourse association and information structure to (1) pinpoint the prosodic features of multi-phrase continuous speech, and (2) simulate native-like expressive speech using corpus of North American and Taiwan L2 English. The bottom-up, additive, data-driven model aims to generate L1-like expressive continuous speech with built-in phonetic and phonological specifications at the lexical level, syntactic/semantic specifications at the next higher phrase and sentence levels, and completed with patterned paragraph associations and prosodic projections of information allocation at higher levels. The hierarchical model successfully allows us to identify L1-L2 differences by prosodic modules/patterns as novel additional features "discourse structure" and "information density" reliably nail down L1-L2 prosodic differences related to phrase association as well as information placement. Our L1 prosodic model with the proposed predictors and optimized model trained from L1 speech corpus showed increase of prediction over existing methods. As a corrective feedback for L2 learners, these predicted L1 prosodic features were compared with a baseline model by objective evaluation (RMS error and correlation) then superimposed onto the L2 speech tokens. Resynthesized L2 tokens were subsequently compared with the original L2 tokens for degrees of perceived accent using subjective evaluation (native-listener perception test). We believe the proposed model can be an effective alternative for implementing computer-assisted language learning (CALL) systems that helps generate L1-like prosody from text, and at the same time serves as corrective feedback for L2 learners.