A study of bias mitigation strategies for speaker recognition. (April 2023)

• Record Type: Journal Article
• Authors: Peri, Raghuveer; Somandepalli, Krishna; Narayanan, Shrikanth
• Is Part Of: Computer speech & language. Volume 79(2023)
• Publication Date: 2023-04
• Subjects: Fairness -- Bias mitigation -- Fairness-utility trade-off -- Speaker verification -- Speaker recognition -- Adversarial training -- Multi task learning; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2022.101481
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: Speaker recognition is increasingly used in several everyday applications including smart speakers, customer care centers and other speech-driven analytics. It is crucial to accurately evaluate and mitigate biases present in machine learning (ML) based speech technologies, such as speaker recognition, to ensure their inclusive adoption. ML fairness studies with respect to various demographic factors in modern speaker recognition systems are lagging compared to other human-centered applications such as face recognition. Existing studies on fairness in speaker recognition systems are largely limited to evaluating biases at specific operating points of the systems, which can lead to false expectations of fairness. Moreover, there are only a handful of bias mitigation strategies developed for speaker recognition systems. In this paper, we systematically evaluate the biases present in speaker recognition systems with respect to gender across a range of system operating points. We also propose adversarial and multi-task learning techniques to improve the fairness of these systems. We show through quantitative and qualitative evaluations that the proposed methods improve the fairness of ASV systems over baseline methods trained using data balancing techniques. We also present a fairness-utility trade-off analysis to jointly examine fairness and the overall system performance. We show that although systems trained using adversarial techniques improve fairness, they are prone to reduced utility. On the other hand, multi-task methods can improve the fairness while retaining the utility. These findings can inform the choice of bias mitigation strategies in the field of speaker recognition. Highlights: Systematic evaluation of gender biases in speaker verification systems. Improvements in fairness through data balancing depend on system's operating point. Both adversarial and multi-task training improve fairness across operating points. Multi-task training retains utility, while adversarial training reduces utility.