Detection of activity and position of speakers by using deep neural networks and acoustic data augmentation. (15th November 2019)

• Record Type: Journal Article
• Authors: Vecchiotti, Paolo; Pepe, Giovanni; Principi, Emanuele; Squartini, Stefano
• Is Part Of: Expert systems with applications. Volume 134(2019)
• Page Start: 53
• Page End: 65
• Publication Date: 2019-11-15
• Subjects: Voice activity detection -- Speaker localization -- Data augmentation -- Multi-room environment -- Deep learning; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2019.05.017
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Innovative neural-based framework for speaker activity and position detection. Exploitation of multiple audio features for improved speech detection. Ad-hoc cascaded CNN architectures for real-world speech detection and localization. Acoustic scene simulation for training data augmentation to enhance the performance. Wide experimental evaluation and relevant improvement with respect to state of art. Abstract: The task of Speaker LOCalization (SLOC) has been the focus of numerous works in the research field, where SLOC is performed on pure speech data, requiring the presence of an Oracle Voice Activity Detection (VAD) algorithm. Nevertheless, this perfect working condition is not satisfied in a real world scenario, where employed VADs do commit errors. This work addresses this issue with an extensive analysis focusing on the relationship between several data-driven VAD and SLOC models, finally proposing a reliable framework for VAD and SLOC. The effectiveness of the approach here discussed is assessed against a multi-room scenario, which is close to a real-world environment. Furthermore, up to the authors' best knowledge, only one contribution proposes a unique framework for VAD and SLOC acting in this addressed scenario; however, this solution does not rely on data-driven approaches. This work comes as an extension of the authors' previous research addressing the VAD and SLOC tasks, by proposing numerous advancements to the original neural network architectures. In details, four different models based on convolutional neural networks (CNNs) are here tested, in order to easily highlight the advantages of the introduced novelties. In addition, two different CNN models go under study for SLOC. Furthermore, training of data-driven models is here improved through a specific data augmentation technique. During this procedure, the room impulse responses (RIRs) of two virtual rooms are generated from the knowledge of the room size, reverberation time and microphones and sources placement. Finally, the only other framework for simultaneous detection and localization in a multi-room scenario is here taken into account to fairly compare the proposed method. As result, the proposed method is more accurate than the baseline framework, and remarkable improvements are specially observed when the data augmentation techniques are applied for both the VAD and SLOC tasks.