Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms. (May 2019) - Williams, Henry A.M.; Jones, Mark H.; Nejati, Mahla; Seabright, Matthew J.; Bell, Jamie; Penhall, Nicky D.; Barnett, Josh J.; Duke, Mike D.; Scarfe, Alistair J.; Ahn, Ho Seok; Lim, JongYoon; MacDonald, Bruce A.
Abstract : As labour requirements in horticultural become more challenging, automated solutions are becoming an effective approach to maintain productivity and quality. This paper presents the design and performance evaluation of a novel multi-arm kiwifruit harvesting robot designed to operate autonomously in pergola style orchards. The harvester consists of four robotic arms that have been designed specifically for kiwifruit harvesting, each with a novel end-effector developed to enable safe harvesting of the kiwifruit. The vision system leverages recent advances in deep neural networks and stereo matching for reliably detecting and locating kiwifruit in real-world lighting conditions. Furthermore, a novel dynamic fruit scheduling system is presented that has been developed to coordinate the four arms throughout the harvesting process. The performance of the harvester has been measured through a comprehensive and realistic field-trial in a commercial orchard environment. The results show that the presented harvester is capable of successfully harvesting 51.0% of the total number of kiwifruit within the orchard with an average cycle time of 5.5s/fruit. Highlights: This paper has reported the design and performance of a novel robotic kiwifruit harvesting system. Detection rate of 89.6% of reachable kiwifruit with a deep network based vision system. Measurement of its in-orchard performance shows that it is capable of picking 51.0% of kiwifruit in the three test orchards. It is estimated that with further development the unit may be capable of harvesting 70% of the kiwifruit