Using speech technology for quantifying behavioral characteristics in peer-led team learning sessions. (November 2017)

• Record Type: Journal Article
• Authors: Dubey, Harishchandra; Sangwan, Abhijeet; Hansen, John H.L.
• Is Part Of: Computer speech & language. Volume 46(2017)
• Page Start: 343
• Page End: 366
• Publication Date: 2017-11
• Subjects: Behavioral speech processing -- Bottleneck features -- Curiosity -- Deep neural network -- Dominance -- Auto-encoder -- Emphasis -- Engagement -- Peer-led team learning -- Speaker diarization -- Small-group conversations; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2017.04.002
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Established CRSS-PLTL corpus (Peer-Led Team Learning) and performed exploratory data analysis. Stacked Denoising Autoencoder-based bottleneck features + Informed HMM-based diarization system. Behavioral Speech Processing for extracting characteristics such as participation, dominance, curiosity (in terms of question inflection), emphasis, engagement. Stacked spectral features were used to train a Deep Neural Network for estimating the fundamental frequency. Fundamental frequency-based method for question inflection detection. Abstract: Peer-Led Team Learning (PLTL) is a learning methodology where a peer-leader co-ordinate a small-group of students to collaboratively solve technical problems. PLTL have been adopted for various science, engineering, technology and maths courses in several US universities. This paper proposed and evaluated a speech system for behavioral analysis of PLTL groups. It could help in identifying the best practices for PLTL. The CRSS-PLTL corpus was used for evaluation of developed algorithms. In this paper, we developed a robust speech activity detection (SAD) by fusing the outputs of a DNN-based pitch extractor and an unsupervised SAD based on voicing measures. Robust speaker diarization system consisted of bottleneck features (from stacked autoencoder) and informed HMM-based joint segmentation and clustering system. Behavioral characteristics such as participation, dominance, emphasis, curiosity and engagement were extracted by acoustic analyses of speech segments belonging to all students. We proposed a novel method for detecting question inflection and performed equal error rate analysis on PLTL corpus. In addition, a robust approach for detecting emphasized speech regions was also proposed. Further, we performed exploratory data analysis for understanding the distortion present in CRSS-PLTL corpus as it was collected in naturalistic scenario. The ground-truth Likert scale ratings were used for capturing the team dynamics in terms of student's responses to a variety of evaluation questions. Results suggested the applicability of proposed system for behavioral analysis of small-group conversations such as PLTL, work-place meetings etc. .