Detection of red and bicoloured apples on tree with an RGB-D camera. (June 2016)

• Record Type: Journal Article
• Authors: Nguyen, Tien Thanh; Vandevoorde, Koenraad; Wouters, Niels; Kayacan, Erdal; De Baerdemaeker, Josse G.; Saeys, Wouter
• Is Part Of: Biosystems engineering. Volume 146(2016:Jun.)
• Page Start: 33
• Page End: 44
• Publication Date: 2016-06
• Subjects: Computer vision -- RGB-D camera -- Fruit detection -- Harvesting robot; Bioengineering -- Periodicals; Agricultural engineering -- Periodicals; Biological systems -- Periodicals; Génie rural -- Périodiques; Systèmes biologiques -- Périodiques; 631
• DOI: 10.1016/j.biosystemseng.2016.01.007
• Languages: English
• ISSNs: 1537-5110

Abstract

Abstract : Recognising and accurately locating fruits on a tree is a critical challenge in developing fruit-by-fruit robotic harvesting. Many researchers have investigated the potential of red, green, blue (RGB) colour imaging for this purpose, but have had limited success due to the occlusion of the target fruits by foliage, branches or other fruits as well as due to the non-uniform and unstructured nature of an orchard environment. Recently, novel, cost-effective camera systems have become available which provide both colour (RGB) and three dimensional (3D) shape information. As these have shown potential for 3D perception for robots operating in unstructured environments, the potential of such an RGB-D camera for the detection and localisation of red and bicoloured apples on tree was investigated in this study. Images were acquired with this camera system in fruit orchards under a light shield blocking direct sunlight, and an algorithm to detect and localise red and bicoloured apples based on colour and shape features was developed. When the algorithm was applied to the data acquired in these orchards, 100% of the fully visible apples and 82% of the partially occluded apples were detected correctly. The location estimation error was below 10 mm in all the coordinate axes of the Cartesian space. This high detection and location accuracy and short processing time (below 1 s for simultaneous detection of 20 apples), makes the developed algorithm suitable for implementation in a robotic harvesting system, and for yield estimation and orchard monitoring. Graphical abstract: Highlights: RGB-D detection and localisation of red an bi-coloured apples on tree. Combines colour segmentation, pixel clustering and RANSAC. 100% of fully visible and 82% of partially occluded apples correctly detected. Location estimation error <10 mm in all the coordinate axes of the Cartesian space. Image processing time below 1 s for simultaneous detection of 20 apples.