What is the best way for extracting meaningful attributes from pictures?. (April 2017)

• Record Type: Journal Article
• Authors: Liu, Liangchen; Wiliem, Arnold; Chen, Shaokang; Lovell, Brian C.
• Is Part Of: Pattern recognition. Volume 64(2017:Apr.)
• Page Start: 314
• Page End: 326
• Publication Date: 2017-04
• Subjects: Visual attribute -- Meaningfulness metric -- Attribute discovering -- Semantic content; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.10.034
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: Automatic attribute discovery methods have gained in popularity to extract sets of visual attributes from images or videos for various tasks. Despite their good performance in some classification tasks, it is difficult to evaluate whether the attributes discovered by these methods are meaningful and which methods are the most appropriate to discover attributes for visual descriptions. In its simplest form, such an evaluation can be performed by manually verifying whether there is any consistent identifiable visual concept distinguishing between positive and negative exemplars labelled by an attribute. This manual checking is tedious, expensive and labour intensive. In addition, comparisons between different methods could also be problematic as it is not clear how one could quantitatively decide which attribute is more meaningful than the others. In this paper, we propose a novel attribute meaningfulness metric to address this challenging problem. With this metric, automatic quantitative evaluation can be performed on the attribute sets; thus, reducing the enormous effort to perform manual evaluation. The proposed metric is applied to some recent automatic attribute discovery and hashing methods on four attribute-labelled datasets. To further validate the efficacy of the proposed method, we conducted a user study. In addition, we also compared our metric with a semi-supervised attribute discover method using the mixture of probabilistic PCA. In our evaluation, we gleaned several insights that could be beneficial in developing new automatic attribute discovery methods. Abstract : Highlights: We proposed a metric to automatically measure visual attribute meaningfulness. The metric is based on subspace interpolation on a decision boundary manifold. Meaningfulness distance is computed via approximated distance on the manifold. An improved metric calibration method is developed based on the in-depth analysis. We present extensive experiments and analysis on four popular attribute datasets.