Attention gated tensor neural network architectures for speech emotion recognition. (January 2022)

• Record Type: Journal Article
• Authors: Pandey, Sandeep Kumar; Shekhawat, Hanumant Singh; Prasanna, S.R.M
• Is Part Of: Biomedical signal processing and control. Volume 71(2022)Part A
• Publication Date: 2022-01
• Subjects: Tensor Factorized Neural Network -- Attention -- CNN -- LSTM -- Speech emotion -- Emo-DB -- IEMOCAP; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.103173
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: In an attempt to make Human-Computer Interactions more natural, we propose the use of Tensor Factorized Neural Networks (TFNN) and Attention Gated Tensor Factorized Neural Network (AG-TFNN) for Speech Emotion Recognition (SER) task. Standard speech representations such as 2D and 3D Mel-Spectrogram and Temporal Modulation Spectrogram is explored to investigate the emotion salient information capturing effectiveness of the Tensor Factorization based architectures. The hidden layers are explained as Deep Tensor Factorization based on the Tucker Decomposition but with a unified discriminative objective function to learn the factor matrices in a discriminative sense. The core tensor produced in each hidden layer is the feature associated with that factorisation layer. Mel Spectrograms are naturally in 2D tensor form, and thus TFNN and AG-TFNN becomes an appropriate choice over baselines such as Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) by providing reduced parameters to be learned and a simple architecture. Experiments conducted on standard emotional speech datasets- Emo-DB and IEMOCAP shows that TFNN and AG-TFNN surpasses the state-of-the-art given by CNN+LSTM combination with fewer number of parameters.