Approaching the accuracy–cost conflict in embedded classification system design. Issue 3 (August 2016)

• Record Type: Journal Article
• Authors: Jensen, Ulf; Kugler, Patrick; Ring, Matthias; Eskofier, Bjoern
• Is Part Of: Pattern analysis and applications. Volume 19:Issue 3(2016:Aug.)
• Page Start: 839
• Page End: 855
• Publication Date: 2016-08
• Subjects: Machine learning -- Real-time systems -- Cost estimation -- Classification system design; Pattern recognition systems -- Periodicals; Pattern perception -- Periodicals; 006.4
• DOI: 10.1007/s10044-015-0503-1
• Languages: English
• ISSNs: 1433-7541

Abstract

Abstract Smart embedded systems often run sophisticated pattern recognition algorithms and are found in many areas like automotive, sports and medicine. The developer of such a system is often confronted with the accuracy–cost conflict as the resulting system should be as accurate as possible while being able to run on resource constraint hardware. This article introduces a method to support the solution of this design conflict with accuracy–cost reports. These reports compare classification systems regarding their classification rate (accuracy) and the mathematical operations and parameters of the working phase (cost). Our method is used to deduce the specific cost of various popular pattern recognition algorithms and to derive the overall cost of a classification system. We also show how our analysis can be used to estimate the computational cost for specific hardware architectures. A software toolbox to create accuracy–cost reports was implemented to facilitate the automatic classification system comparison with the presented methodology. The software is available for download and as supplementary material. We performed different experiments on synthetic and real-world data to underline the value of this analysis. Accurate and computationally cheap classification systems were easily identified. We were even able to find a better implementation candidate in an existing embedded classification problem. This work is the first step towards a comprehensive support tool for the design of embedded classification systems.