Product verification using OCR classification and Mondrian conformal prediction. (February 2022)

• Record Type: Journal Article
• Authors: Oucheikh, Rachid; Pettersson, Tobias; Löfström, Tuwe
• Is Part Of: Expert systems with applications. Volume 188(2022)
• Publication Date: 2022-02
• Subjects: OCR classification -- Retail product verification -- Mondrian conformal prediction -- Smart self-checkout system; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115942
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: The retail sector is undergoing an apparent digital transformation that completely revolutionises shopping operations. To stay competitive, retailer stakeholders are forced to rethink and improve their business models to provide an attractive personalised experience to consumers. The self-service checkout process is at the heart of this transformation and should be designed to identify the products accurately and detect any possible anomalous behaviour. In this paper, we introduce a product verification system based on OCR classification and Mondrian conformal prediction. The proposed system includes three components: OCR reading, text classification and product verification. By using image data from existing grocery stores, the system can detect anomalies with high performance, even when there is partial text information on the products. This makes the system applicable for reducing shrinkage loss (caused, for example, by employee theft or shoplifting) in grocery stores by identifying fraudulent behaviours such as barcode switching and miss-scan. Additionally, OCR reading with NLP classification shows that it is in itself a powerful classifier of products. Highlights: We propose an OCR-based product verification system for smart retail self-checkout. The system consists of OCR reading, text classification and product verification. The verification is performed using Mondrian conformal prediction. The product verification system achieves high performance for anomaly detection.