String representations and distances in deep Convolutional Neural Networks for image classification. (June 2016)

• Record Type: Journal Article
• Authors: Barat, Cécile; Ducottet, Christophe
• Is Part Of: Pattern recognition. Volume 54(2016:Jun.)
• Page Start: 104
• Page End: 115
• Publication Date: 2016-06
• Subjects: Convolutional Neural Network -- String representation -- Edit distance -- Image classification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.01.007
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: Recent advances in image classification mostly rely on the use of powerful local features combined with an adapted image representation. Although Convolutional Neural Network (CNN) features learned from ImageNet were shown to be generic and very efficient, they still lack of flexibility to take into account variations in the spatial layout of visual elements. In this paper, we investigate the use of structural representations on top of pretrained CNN features to improve image classification. Images are represented as strings of CNN features. Similarities between such representations are computed using two new edit distance variants adapted to the image classification domain. Our algorithms have been implemented and tested on several challenging datasets, 15Scenes, Caltech101, Pascal VOC 2007 and MIT indoor. The results show that our idea of using structural string representations and distances clearly improves the classification performance over standard approaches based on CNN and SVM with linear kernel, as well as other recognized methods of the literature. Abstract : Highlights: A structural representation of images on top of CNN features is proposed. Images are represented as strings to integrate spatial relationships. We introduce tailored string edit distances to compare images represented as strings. Experiments show that our structural approach is more powerful than existing ones. It also outperforms state-of-the-art CNN-based classification methods.