Improving the Quality of Sound Recovered Using the Visual Microphone with Frame-wise Image Denoising Preprocessing. (August 2020)

• Record Type: Journal Article
• Authors: Choong, Ren-Jun; Yap, Wun-She; Chai Hum, Yan; Kai Tee, Yee
• Is Part Of: Journal of physics. Volume 1627(2020)
• Publication Date: 2020-08
• Subjects: Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/1627/1/012024
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: The visual microphone is a passive remote sound recovery technique by analysing subtle vibrations in a video of an object vibrating due to sound. Multiple research endeavours have been done in improving the quality of the recovered sound, but so far none has been done on investigating the effect of denoising the videos frame by frame prior to the sound recovery using the visual microphone. This work fills in that gap, and evaluates the quality of the recovered sound based on human intelligibility metrics and simple signal-to-noise ratio metrics. Our best results indicated performing image denoising before sound recovery can reduce the noise power by up to 47.16 %, increase the intelligibility by 9.11 % and signal-to-noise ratio by 17.97 %. It is important to perform image denoising before sound recovery when the recorded videos are very noisy.