A model of synesthetic metaphor interpretation based on cross-modality similarity. (November 2019)

• Record Type: Journal Article
• Authors: Su, Chang; Wang, Xiaomei; Wang, Zita; Chen, Yijiang
• Is Part Of: Computer speech & language. Volume 58(2019)
• Page Start: 1
• Page End: 16
• Publication Date: 2019-11
• Subjects: Synesthetic metaphor -- Semantic knowledge -- Cross-modality similarity; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2019.03.003
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: Synesthesia focuses on the cross-modality connections between different perceptual modalities (visual, auditory, olfactory, gustatory, and haptic). It presents the aesthetics and creativity of our language and cognitive systems. A model that automatically interprets synesthetic metaphor is proposed in this paper. The interpretation model can learn the semantic knowledge of a feature, and simulate the cross-modality semantic similarity. The semantic knowledge has three parts: feature, the perceptual modality of the feature, and its sentimental orientation. They are used for supervising feature generation and expansion. By multi-step synonyms expansion, our model explores the cross-modality associations between the source and target domains which belong to different modalities. The appropriate feature is selected by a heuristic way to describe the latent meaning of the target domain in the current context. The experimental result shows that our model can fully consider the semantic knowledge and embody the cross-modality relations. It achieves an accuracy of 84%. Highlights: A model can automatically interpret synesthetic metaphor. Introduce and analyze the semantic knowledge of the feature, including feature, perceptual modality and sentiment. A heuristic method for feature expansion and generation. Cross-modality semantic similarity is well simulated and measured. Interpretation model is interpretable and observable.