A multimodal query expansion based on genetic programming for visually-oriented e-commerce applications. Issue 5 (September 2016)

• Record Type: Journal Article
• Authors: Saraiva, Patrícia C.; Cavalcanti, João M.B.; de Moura, Edleno S.; Gonçalves, Marcos A.; Torres, Ricardo da S.
• Is Part Of: Information processing & management. Volume 52:Issue 5(2016:Sep.)
• Page Start: 783
• Page End: 800
• Publication Date: 2016-09
• Subjects: Content-based image retrieval -- Multimodal query expansion -- Genetic programming -- E-commerce; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2016.03.001
• Languages: English
• ISSNs: 0306-4573

Abstract

Highlights: An application of Genetic Programming where the results are better than previous results found in literature. A comparison of a Genetic Programming solution with two other learning-to-rank techniques: RankSVM and Random Forests. A multimodal expansion uses the initial query to infer other attributes that are relevant to the query. A new approach for visually-oriented e-commerce applications. A solution useful when the user is searching for products such as clothing, shoes, handbags, watches, and accessories. A solution that allows finding relevant products related to an image query even though their image representation is not similar to the query. Abstract: We present a novel multimodal query expansion strategy, based on genetic programming (GP), for image search in visually-oriented e-commerce applications. Our GP-based approach aims at both: learning to expand queries with multimodal information and learning to compute the "best" ranking for the expanded queries. However, different from previous work, the query is only expressed in terms of the visual content, which brings several challenges for this type of application. In order to evaluate the effectiveness of our method, we have collected two datasets containing images of clothing products taken from different online shops. Experimental results indicate that our method is an effective alternative for improving the quality of image search results when compared to a genetic programming system based only on visual information. Our method can achieve gains varying from 10.8% against the strongest learning-to-rank baseline to 54% against an adhoc specialized solution for the particular domain at hand.