Collaborative Research on Mouth Shape and Lyrics in Singing Practice Based on Image Processing. (28th January 2022)

• Record Type: Journal Article
• Authors: Xu, Lujia; Chen, Chen
• Other Names: Ding Baiyuan Academic Editor.
• Is Part Of: Computational intelligence and neuroscience. Volume 2022(2022)
• Publication Date: 2022-01-28
• Subjects: Neurosciences -- Data processing -- Periodicals; Computational intelligence -- Periodicals; Computational neuroscience -- Periodicals; 612.80285
• DOI: 10.1155/2022/5138442
• Languages: English
• ISSNs: 1687-5265

Abstract

Abstract : Image processing is a mainstream processing method. When people enjoy artists' singing videos, there will be a problem that the subtitles of the lyrics are out of sync with the singer's mouth shape. This problem needs to be solved using image processing technology, letting the computer realize lip-reading recognition function and correct the mouth shape and lyrics subtitles in the image according to the extracted lip-reading data, so that the mouth shape and lyrics in singing practice can be synchronized. Lip-reading information can effectively improve the accuracy of language cognition, save part of capital and manpower investment, and make viewers get a good audio-visual interactive experience. The results show the following: (1) After the UI test, the system user interface function design is reasonable and there is no bad BUG. We can find that the average processing time of each frame is 628 ms, the system performance evaluation is good, and the success rate can be as high as 98.80%. 0.36724 s is the average time for each step when the system processes the image. (2) The human image can basically identify the portrait area and lip area from various angles. (3) Compared with DCT and DWT, the recognition rate of the two cascade lip region feature extraction methods is improved by nearly 10%, and the feature vector dimension is reduced by nearly 65%. (4) Classify the mouth shape more finely and optimize the image of the tester's mouth shape to make the mouth shape closer to the standard mouth shape. (5) After systematic correction of mouth shape and subtitles, the success rate is higher than 90%. Finally, we can find that the running effect is good and the method has achieved high results, which can carry out the details of the next optimization work.