Intelligent trademark recognition and similarity analysis using a two-stage transfer learning approach. (April 2022)

• Record Type: Journal Article
• Authors: Trappey, Amy J.C.; Trappey, Charles V.; Lin, Eason
• Is Part Of: Advanced engineering informatics. Volume 52(2022)
• Publication Date: 2022-04
• Subjects: Triplet neural network -- Transfer learning -- Trademark (TM) infringement -- Yolo v4; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2022.101567
• Languages: English
• ISSNs: 1474-0346

Abstract

Highlights: Develop a two-stage transfer learning system for IP, especially registered trademark, image protections. The logo detection and localization model for cropping TM like images from complex online merchandise photos. Triplet convolutional neural network model is used for intelligent TM similarity analysis with high matching rate. The system performance is quantitatively tested using FlickrLogos-32 dataset and EC site demonstration. Abstract: The ubiquitous and fast-growing e-marketplaces are causing serious concerns of unauthorized trademark usages, especially their pictorial images. There is a significant need for e-commerce service intermediaries and trading sites to actively check whether product images posted online are not infringing upon others' intellectual property rights (IPRs). To ensure registered trademark (TM) or copyrighted logo-image (logo) protection, this research develops an intelligent system that can detect, locate, and crop (cut) logos posted online and check whether they are substantially or deceptively similar to registered TM logos. This research integrates two deep learning models to achieve the research goal. The first is the logo detection and localization model for cropping trademark like images from complex online merchandise photos, which could have images of many product views and even worn and displayed on a human model. Only the cropped logo image is needed for comparison to a database of registered trademarks. The second model performs TM similarity analysis using the cropped logos compared to the published TM images. Yolo v4 is adopted as the general logo locator for intelligent logo image cropping. The triplet convolutional neural network model is used to fine-tuned for intelligent trademark similarity analysis. The models are trained with an image dataset, combining image samples from LogoDet-3k and images found through web search. The system performance is quantitatively tested using the FlickrLogos-32 dataset and consumer product images extracted from an e-commerce platform. The test results of the system achieve high precision (0.91) for trademark class matching.