Incremental transfer learning for video annotation via grouped heterogeneous sources. Issue 1 (20th January 2020)

• Record Type: Journal Article
• Authors: Wang, Han; Song, Hao; Wu, Xinxiao; Jia, Yunde
• Is Part Of: IET computer vision. Volume 14:Issue 1(2020)
• Page Start: 26
• Page End: 35
• Publication Date: 2020-01-20
• Subjects: video signal processing -- Internet -- learning (artificial intelligence); source domain data -- entire target domain videos -- source group -- group weights -- incremental learning process -- negative transfer -- large-scale consumer video datasets -- video annotation -- grouped heterogeneous sources -- heterogeneous knowledge -- internet datasets -- tedious labelling efforts -- expensive labelling efforts -- incremental transfer learning framework -- heterogeneous source knowledge -- annotation model -- transfer learning process -- web images -- existing action videos -- labelled static motion information; Computer vision -- Periodicals; Pattern recognition systems -- Periodicals; 006.37
• DOI: 10.1049/iet-cvi.2018.5730
• Languages: English
• ISSNs: 1751-9632

Abstract

Abstract : Here, the authors focus on incrementally acquiring heterogeneous knowledge from both internet and publicly available datasets to reduce the tedious and expensive labelling efforts required in video annotation. An incremental transfer learning framework is presented to integrate heterogeneous source knowledge and update the annotation model incrementally during the transfer learning process. Under this framework, web images and existing action videos form the source domain to provide labelled static and motion information of the target domain videos, respectively. Moreover, according to the semantic of the source domain data, all the source domain data are partitioned into several groups. Different from traditional methods, which compare the entire target domain videos with each source group from the source domain, the authors treat the group weights as sample‐specific variables and optimise them along with new adding data. Two regularisers are used to prevent the incremental learning process from negative transfer. Experimental results on the two large‐scale consumer video datasets (i.e. multimedia event detection (MED) and Columbia consumer video (CCV)) show the effectiveness of the proposed method.