Towards understanding and mitigating unintended biases in language model-driven conversational recommendation. Issue 1 (January 2023)

• Record Type: Journal Article
• Authors: Shen, Tianshu; Li, Jiaru; Bouadjenek, Mohamed Reda; Mai, Zheda; Sanner, Scott
• Is Part Of: Information processing & management. Volume 60:Issue 1(2023)
• Publication Date: 2023-01
• Subjects: Conversational recommendation systems -- BERT -- Contextual language models -- Bias and discrimination; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2022.103139
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Conversational Recommendation Systems (CRSs) have recently started to leverage pretrained language models (LM) such as BERT for their ability to semantically interpret a wide range of preference statement variations. However, pretrained LMs are prone to intrinsic biases in their training data, which may be exacerbated by biases embedded in domain-specific language data (e.g., user reviews) used to fine-tune LMs for CRSs. We study a simple LM-driven recommendation backbone (termed LMRec) of a CRS to investigate how unintended bias — i.e., bias due to language variations such as name references or indirect indicators of sexual orientation or location that should not affect recommendations — manifests in substantially shifted price and category distributions of restaurant recommendations. For example, offhand mention of names associated with the black community substantially lowers the price distribution of recommended restaurants, while offhand mentions of common male-associated names lead to an increase in recommended alcohol-serving establishments. While these results raise red flags regarding a range of previously undocumented unintended biases that can occur in LM-driven CRSs, there is fortunately a silver lining: we show that train side masking and test side neutralization of non-preferential entities nullifies the observed biases without significantly impacting recommendation performance. Highlights: We study bias in a simple language model-driven recommender (LMRec). LMRec demonstrates substantial price and category shifts based on race and gender. LMRec appears to pick up on indirect mentions of homosexual relations. LMRec demonstrates substantial price shift based on location and religion mentions. Train side masking and test side neutralization maintain accuracy and nullify bias.