Room-localized spoken command recognition in multi-room, multi-microphone environments. (November 2017)

• Record Type: Journal Article
• Authors: Rodomagoulakis, Isidoros; Katsamanis, Athanasios; Potamianos, Gerasimos; Giannoulis, Panagiotis; Tsiami, Antigoni; Maragos, Petros
• Is Part Of: Computer speech & language. Volume 46(2017)
• Page Start: 419
• Page End: 443
• Publication Date: 2017-11
• Subjects: Smart homes -- Distant speech recognition -- Speech activity detection -- Keyword spotting -- Multichannel processing -- Decision fusion -- Beamforming -- Channel selection; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2017.02.004
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Always-listening recognition pipeline for multi-room smart spaces. Room-localized operation based on multi-room speech activity detection. Channel selection and decision fusion approaches in all pipeline components. Robust acoustic modeling based on far-field data simulation and per-channel adaptation. Systematic pipeline evaluation and optimization on both simulated and real corpora. Abstract: The paper focuses on the design of a practical system pipeline for always-listening, far-field spoken command recognition in everyday smart indoor environments that consist of multiple rooms equipped with sparsely distributed microphone arrays. Such environments, for example domestic and multi-room offices, present challenging acoustic scenes to state-of-the-art speech recognizers, especially under always-listening operation, due to low signal-to-noise ratios, frequent overlaps of target speech, acoustic events, and background noise, as well as inter-room interference and reverberation. In addition, recognition of target commands often needs to be accompanied by their spatial localization, at least at the room level, to account for users in different rooms, providing command disambiguation and room-localized feedback. To address the above requirements, the use of parallel recognition pipelines is proposed, one per room of interest. The approach is enabled by a room-dependent speech activity detection module that employs appropriate multichannel features to determine speech segments and their room of origin, feeding them to the corresponding room-dependent pipelines for further processing. These consist of the traditional cascade of far-field spoken command detection and recognition, the former based on the detection of "activating" key-phrases. Robustness to the challenging environments is pursued by a number of multichannel combination and acoustic modeling techniques, thoroughly investigated in the paper. In particular, channel selection, beamforming, and decision fusion of single-channel results are considered, with the latter performing best. Additional gains are observed, when the employed acoustic models are trained on appropriately simulated reverberant and noisy speech data, and are channel-adapted to the target environments. Further issues investigated concern the inter-dependencies of the various system components, demonstrating the superiority of joint optimization of the component tunable parameters over their separate or sequential optimization. The proposed approach is developed for the Greek language, exhibiting promising performance in real recordings in a four-room apartment, as well as a two-room office. For example, in the latter, a 76.6% command recognition accuracy is achieved on a speaker-independent test, employing a 180-sentence decoding grammar. This result represents a 46% relative improvement over conventional beamforming.