Image Classification Benchmark (ICB). (15th March 2020)

• Record Type: Journal Article
• Authors: Caldeira, Manuel; Martins, Pedro; Costa, Rogério Luís C.; Furtado, Pedro
• Is Part Of: Expert systems with applications. Volume 142(2020)
• Publication Date: 2020-03-15
• Subjects: Benchmark ecosystem -- Image classification -- CNN -- Food recognition; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2019.112998
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: An image benchmark ecosystem to easily configure and run algorithms and workflows. Performance metrics can be measured and compared to the ones of state-of-the-art systems. A default dataset is provided, but it can be replaced or added to support specific experiments. A step-by-step comparison on the use of three convolution neural networks in image food recognition. Abstract: During any investigative work, it is necessary to confront our solutions to the already existing ones. This requires much work, namely to try to recreate the competing solutions and to use the same experimental conditions in order to obtain an objective comparison. Benchmarking is an activity of comparing performance metrics of systems in order to rank them or the activity of comparing a specific system to state-of-the-art alternatives. It would be convenient to have an image benchmarking ecosystem, not only to evaluate a personal solution but also to compare it with other previously proposed solutions. Upon designing a new or improved image classification pipeline, a convolution neural network or a specific algorithm modifying some detail, a user can add it to the benchmarking ecosystem and get a report on the performance of the solution, which can be compared to other solutions that were previously benchmarked under the same conditions. In this paper we describe the ICB - "Image Classification Benchmark" -, a prototype of a benchmarking ecosystem created to enable this vision. Besides describing how it works and how it was made flexible to incorporate any algorithm, we apply it to a specific comparison, as a proof-of-concept.