Automatic Piano Music Transcription Using Audio‐Visual Features. Issue 3 (1st July 2015)

• Record Type: Journal Article
• Authors: Wan, Yulong; Wang, Xianliang; Zhou, Ruohua; Yan, Yonghong
• Is Part Of: Chinese journal of electronics. Volume 24:Issue 3(2015)
• Page Start: 596
• Page End: 603
• Publication Date: 2015-07-01
• Subjects: audio‐visual systems -- computer vision -- image filtering -- matched filters -- music -- musical instruments -- object detection -- object tracking; automatic piano music transcription -- audio‐visual features -- music instruments -- piano‐specific transcription system -- onset detection method -- specific spectrum envelope matched filter -- multiple frequency bands -- computer‐vision method -- audio‐only piano music transcription enhancement -- MIDI aligned piano sound database -- MAPS database -- piano keyboard -- pianist hand position tracking -- music information retrieval evaluation exchange -- MIREX; Electronics -- Periodicals; Electronics -- China -- Periodicals; Electronics; China; Periodicals; 621.38105
• DOI: 10.1049/cje.2015.07.027
• Languages: English
• ISSNs: 1022-4653

Abstract

Abstract : The performance of automatic music transcription seems to have reached a limit over the last decade, and a promising direction of improvements could be to incorporate music instruments' specific parameters. We propose a novel piano‐specific transcription system, using both audio and visual features for the first time. Contribution of the paper mainly includes two parts: A new onset detection method is proposed using a specific spectrum envelope matched filter on multiple frequency bands. A computer‐vision method is proposed to enhance audio‐only piano music transcription, through tracking the positions of the pianist's hands on the piano keyboard. Based on the MIDI Aligned piano sounds (MAPS) database and a self recorded video database, we carried out comparable experiments for audio‐only onset detection and overall system, respectively. The performance was compared with the best piano transcription system in Music information retrievalevaluation exchange (MIREX), and the results showed that the proposed system out performs the state‐of‐art method substantially.