Audiovisual interaction with rate-varying signals. (November 2022)

• Record Type: Journal Article
• Authors: Yi, Long; Sekuler, Robert
• Is Part Of: I-Perception. Volume 13:Number 6(2022)
• Publication Date: 2022-11
• Subjects: audiovisual interaction -- confusion matrix -- similarity space -- temporal correlation -- rate perception; Perception -- Periodicals; Visual perception -- Periodicals; Perception; Perception; Electronic journals; Periodical; Fulltext; Internet Resources; Periodicals; Periodicals; 153.7
• DOI: 10.1177/20416695221116653
• Languages: English
• ISSNs: 2041-6695

Abstract

A task-irrelevant, amplitude-modulating sound influences perception of a size-modulating visual stimulus. To probe the limits of this audiovisual interaction we vary the second temporal derivative of object size and of sound amplitude. In the study's first phase subjects see a visual stimulus size-modulating withf ″ ( x ) > 0, 0, or <0, and judge each one's rate as increasing, constant, or decreasing. Visual stimuli are accompanied by a steady, non-modulated auditory stimulus. The novel combination of multiple stimuli and multi-alternative responses allows subjects' similarity space to be estimated from the stimulus-response confusion matrix. In the study's second phase, rate-varying visual stimuli are presented in concert with auditory stimuli whose second derivative also varied. Subjects identified each visual stimuli as one of the three types, while trying to ignore the accompanying sound. Unlike some previous results withf ″ ( x ) fixed at 0, performance benefits relatively little when visual and auditory stimuli share the same directional change in modulation. However, performance does drop when visual and auditory stimului differ in their directions of rate change. Our task's computational demands may make it particularly vulnerable to the effects of a dynamic task-irrelevant stimulus.