An automated approach for the recognition of intended messages in grouped bar charts. (23rd June 2019)

• Record Type: Journal Article
• Authors: Burns, Richard; Carberry, Sandra; Elzer Schwartz, Stephanie
• Is Part Of: Computational intelligence. Volume 35:Number 4(2019)
• Page Start: 955
• Page End: 1002
• Publication Date: 2019-06-23
• Subjects: automated reasoning -- Bayesian networks -- caption processing -- grouped bar charts -- user modeling; Artificial intelligence -- Periodicals; Computational linguistics -- Periodicals; 006.3
• DOI: 10.1111/coin.12227
• Languages: English
• ISSNs: 0824-7935

Abstract

Abstract: Information graphics (bar charts, line graphs, grouped bar charts, etc) often appear in popular media such as newspapers and magazines. In most cases, the information graphic is intended to convey a high‐level message. This message plays a role in facilitating the discourse purpose of the document but is seldom repeated in the document's text, headlines, or captions. We present a methodology and an implemented system for recognizing the intended message of a grouped bar chart. The recognition system relies on the following components: (1) a linguistic classifier that processes text in the graphic and predicts the most linguistically salient entity from those that are mentioned in text, (2) a cognitive model that estimates the relative perceptual effort required for an individual to recognize some high‐level message in a graph, and (3) a Bayesian network that captures the probabilistic relationship between the high‐level intended message of a graphic and its communicative signals. This research contributes to three applications: accessibility of information graphics for sight‐impaired individuals, retrieval of information graphics from a digital library, and summarization of multimodal documents.