Monocular positioning of sweet peppers: An instance segmentation approach for harvest robots. (August 2020)

• Record Type: Journal Article
• Authors: Chen, Cheng; Li, Bo; Liu, Jiaxiang; Bao, Tong; Ren, Ni
• Is Part Of: Biosystems engineering. Volume 196(2020)
• Page Start: 15
• Page End: 28
• Publication Date: 2020-08
• Subjects: Harvesting robots -- Positioning -- Instance segmentation -- Multi-task network -- Convolutional neural network.; Bioengineering -- Periodicals; Agricultural engineering -- Periodicals; Biological systems -- Periodicals; Génie rural -- Périodiques; Systèmes biologiques -- Périodiques; 631
• DOI: 10.1016/j.biosystemseng.2020.05.005
• Languages: English
• ISSNs: 1537-5110

Abstract

Abstract : Accurate positioning of fruit is a key issue that has attracted much attention in the field of harvest robots. The complex environment and close proximity make the perception of dense crops in greenhouses a challenging problem. Different from various solutions proposed involving special equipment or other auxiliary information, we propose a novel positioning approach based on instance segmentation using a monocular RGB camera. To achieve high position accuracy, we first design a deep convolutional neural network (CNN) in a multi-task framework to export a binary segmentation map and an embedded feature map. To solve the problem of performance degradation in the intersection-over-union (IoU) for the binary segmentation task caused by multi-task optimisation, the encoder part of our network is redesigned on the basis of a Visual Geometry Group network with 16 convolutional layers (VGG-16). Then, mean-shift clustering is used to achieve instance segmentation. Finally, a contour-finding algorithm is presented for outlining fruit without the help of any contextual information. Based on these contours, the five fruits with the largest contour areas are selected as the targets for positioning. We verify our method on a public sweet pepper dataset and achieve competitive results. Divided by the radius of the fruit, the average position error for the first target in the harvesting order is 0.18, which shows that our method outperforms the semantic segmentation method. For the first five targets in the harvesting order, this index is less than 0.3 on average, similar to that of the semantic segmentation method with only one target output. Highlights: An approach is presented to detect and locate sweet peppers in monocular RGB images. The segmentation approach for positioning does not require context information. Multiple harvesting targets are assessed simultaneously in an image. Average error – in pixels - of fruit centre position over fruit radius was 18.3%.