Image-based size estimation of broccoli heads under varying degrees of occlusion. (August 2021)

• Record Type: Journal Article
• Authors: Blok, Pieter M.; van Henten, Eldert J.; van Evert, Frits K.; Kootstra, Gert
• Is Part Of: Biosystems engineering. Volume 208(2021)
• Page Start: 213
• Page End: 233
• Publication Date: 2021-08
• Subjects: Size estimation -- Occlusion -- Deep-learning -- Agriculture -- Data set -- Computer vision; Bioengineering -- Periodicals; Agricultural engineering -- Periodicals; Biological systems -- Periodicals; Génie rural -- Périodiques; Systèmes biologiques -- Périodiques; 631
• DOI: 10.1016/j.biosystemseng.2021.06.001
• Languages: English
• ISSNs: 1537-5110

Abstract

Abstract : The growth and the harvestability of a broccoli crop is monitored by the size of the broccoli head. This size estimation is currently done by humans, and this is inconsistent and expensive. The goal of our work was to develop a software algorithm that can estimate the size of field-grown broccoli heads based on RGB-Depth (RGB-D) images. For the algorithm to be successful, the problem of occlusion must be solved, which is the partial visibility of the broccoli head due to overlapping leaves. This partial visibility causes sizing errors. In this research, we studied the use of deep-learning algorithms to deal with occlusions. We specifically applied the Occlusion Region-based Convolutional Neural Network (ORCNN) that segmented both the visible and the amodal region of the broccoli head (which is the visible and the occluded region combined). We hypothesised that ORCNN, with its amodal segmentation, can improve the size estimation of occluded broccoli heads. The ORCNN sizing method was compared with a Mask R–CNN sizing method that only used the visible broccoli region to estimate the size. The sizing performance of both methods was evaluated on a test set of 487 broccoli images with systematic levels of leaf occlusion. With a mean sizing error of 6.4 mm, ORCNN outperformed Mask R–CNN, which had a mean sizing error of 10.7 mm. Furthermore, ORCNN had a significantly lower absolute sizing error on 161 heavily occluded broccoli heads with an occlusion rate between 50% and 90%. Our software and data set are available on https://git.wur.nl/blok012/sizecnn. Highlights: The image-based size estimation of broccoli heads was studied. A data set of 2560 broccoli images was acquired with systematic levels of occlusion. Two deep-learning methods were tested on their ability to size occluded broccolis. One deep-learning method used a dual segmentation to better deal with the occlusion. The software and the data set have been made available.