Multivariate mutual information for audio video fusion. Issue 7 (October 2016)

• Record Type: Journal Article
• Authors: Dilpazir, Hammad; Muhammad, Zia; Minhas, Qurratulain; Ahmed, Faheem; Malik, Hafiz; Mahmood, Hasan
• Is Part Of: Signal, image and video processing. Volume 10:Issue 7(2016)
• Page Start: 1265
• Page End: 1272
• Publication Date: 2016-10
• Subjects: Data fusion -- Audio video processing -- Information theory -- Multimodal data processing -- Multimedia processing -- Multivariate mutual information -- Auditory–visual integration; Signal processing -- Digital techniques -- Periodicals; Image processing -- Digital techniques -- Periodicals; Digital video -- Periodicals; 621.3822
• DOI: 10.1007/s11760-016-0892-7
• Languages: English
• ISSNs: 1863-1703

Abstract

Abstract Information fusion is one of the essential part of distributed wireless sensor networks as well as perceptual user interfaces. Irrelevant and redundant data severely affect the performance of the information fusion process. In this paper, a method based on multivariate mutual information is presented to validate the acceptability of data from two sources (visual and auditory). The audiovisual information is fused to observe theventriloquism effect to validate the algorithm. Unlike the preceding algorithms, this framework does not require any preprocessing such as automatic face recognition. Moreover, statistical modeling or feature extraction and learning algorithms are not required to extract the maximum information regions. The results for various cases, containing a single speaker as well as a group of speakers, are also presented.