Semantic Labeling of Nonspeech Audio Clips. (1st March 2010)

• Record Type: Journal Article
• Authors: Ma, Xiaojuan; Fellbaum, Christiane; Cook, Perry
• Other Names: Chen Liming Academic Editor.
• Is Part Of: EURASIP journal on audio, speech, and music processing. Volume 2010(2010)
• Publication Date: 2010-03-01
• Subjects: Sound -- Recording and reproducing -- Digital techniques -- Periodicals; Computer sound processing -- Periodicals; Computer sound processing; Sound -- Recording and reproducing -- Digital techniques; Periodicals; Electronic journal; Electronic journals; 620.2
• DOI: 10.1155/2010/404860
• Languages: English
• ISSNs: 1687-4714

Abstract

Abstract : Human communication about entities and events is primarily linguistic in nature. While visual representations of information are shown to be highly effective as well, relatively little is known about the communicative power of auditory nonlinguistic representations. We created a collection of short nonlinguistic auditory clips encoding familiar human activities, objects, animals, natural phenomena, machinery, and social scenes. We presented these sounds to a broad spectrum of anonymous human workers using Amazon Mechanical Turk and collected verbal sound labels. We analyzed the human labels in terms of their lexical and semantic properties to ascertain that the audio clips do evoke the information suggested by their pre-defined captions. We then measured the agreement with the semantically compatible labels for each sound clip. Finally, we examined which kinds of entities and events, when captured by nonlinguistic acoustic clips, appear to be well-suited to elicit information for communication, and which ones are less discriminable. Our work is set against the broader goal of creating resources that facilitate communication for people with some types of language loss. Furthermore, our data should prove useful for future research in machine analysis/synthesis of audio, such as computational auditory scene analysis, and annotating/querying large collections of sound effects.