Computer vision based detection of external defects on tomatoes using deep learning. (February 2020)

• Record Type: Journal Article
• Authors: da Costa, Arthur Z.; Figueroa, Hugo E.H.; Fracarolli, Juliana A.
• Is Part Of: Biosystems engineering. Volume 190(2020)
• Page Start: 131
• Page End: 144
• Publication Date: 2020-02
• Subjects: Defect sorting -- Deep learning -- Sorting machines -- Computer vision; Bioengineering -- Periodicals; Agricultural engineering -- Periodicals; Biological systems -- Periodicals; Génie rural -- Périodiques; Systèmes biologiques -- Périodiques; 631
• DOI: 10.1016/j.biosystemseng.2019.12.003
• Languages: English
• ISSNs: 1537-5110

Abstract

Abstract : Sorting machines use computer vision (CV) to separate food items based on various attributes. For instance, sorting based on size and colour are commonly used in commercial machines. However, detecting external defects using CV remains an open problem. This paper presents an experimental contribution to external defect detection using deep learning. An uncensored dataset with 43, 843 images including external defects was built during this study. The dataset is heavily imbalanced towards the healthy class, and it is available online. Deep residual neural network (ResNet) classifiers were trained that are capable of detecting external defects using feature extraction and fine-tuning. The results show that fine-tuning outperformed feature extraction, revealing the benefit of training additional layers when sufficient data samples are available. The best model was a ResNet50 with all its layers fine-tuned. This model achieved an average precision of 94.6 % on the test set. The optimal classifier had a recall of 86.6 % while maintaining a precision of 91.7 % . The posterior class-conditional distributions of the classifier scores showed that the key to classifier success lies in its almost ideal healthy class distribution. The results also explain why the model does not confuse stems/calyxes with external defects. The best model constitutes a milestone for detecting external defects using CV. Because deep learning does not require feature engineering or prior knowledge about the dataset content, the methodology may also work well with other foods. Highlights: Deep learning detects different types of external defects without feature engineering. Fine tuning outperformed feature extraction in external defect detection. The best model has Average Precision = 94.6 %. The best model has 86.6 % Recall and 91.7 % Precision at the optimal threshold. The best model does not confuse external defects with stems/calyxes.