Generating visual explanations with natural language. Issue 4 (19th December 2021)

• Record Type: Journal Article
• Authors: Hendricks, Lisa Anne; Rohrbach, Anna; Schiele, Bernt; Darrell, Trevor; Akata, Zeynep
• Other Names: Gunning Dave guestEditor.; Vorm Eric guestEditor.; Wang Jennifer Yunyan guestEditor.; Turek Matt guestEditor.
• Is Part Of: Applied AI Letters. Volume 2:Issue 4(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-12-19
• Subjects: computer vision -- explanations -- fine‐grained classification -- natural language; 006.3
• DOI: 10.1002/ail2.55
• Languages: English
• ISSNs: 2689-5595

Abstract

Abstract: We generate natural language explanations for a fine‐grained visual recognition task. Our explanations fulfill two criteria. First, explanations are class discriminative, meaning they mention attributes in an image which are important to identify a class. Second, explanations are image relevant, meaning they reflect the actual content of an image. Our system, composed of an explanation sampler and phrase‐critic model, generates class discriminative and image relevant explanations. In addition, we demonstrate that our explanations can help humans decide whether to accept or reject an AI decision.