Instance segmentation of apple flowers using the improved mask R–CNN model. (May 2020)

• Record Type: Journal Article
• Authors: Tian, Yunong; Yang, Guodong; Wang, Zhe; Li, En; Liang, Zize
• Is Part Of: Biosystems engineering. Volume 193(2020)
• Page Start: 264
• Page End: 278
• Publication Date: 2020-05
• Subjects: Apple flower images acquisition -- Image augmentation -- Deep learning -- MASU R–CNN -- Instance segmentation; Bioengineering -- Periodicals; Agricultural engineering -- Periodicals; Biological systems -- Periodicals; Génie rural -- Périodiques; Systèmes biologiques -- Périodiques; 631
• DOI: 10.1016/j.biosystemseng.2020.03.008
• Languages: English
• ISSNs: 1537-5110

Abstract

Abstract : Flower and fruitlet thinning can be an effective method of improving the yield and quality of fruit. Automatic detection flowers and fruits at different growth stages is essential for the intelligent management of apple orchards. The further segmentation of blossom areas contributes to extracting detailed growth information of apple flowers. However, the precise detection and segmentation of blossom images is yet to be fully accomplished. An instance segmentation model which improves Mask Scoring R–CNN with a U-Net backbone (MASU R–CNN) is proposed for the detection and segmentation of apple flowers with three different levels of growth status: bud, semi-open and fully open. The foreground and background of apple flower images were combined based on the growth characteristics of apple flowers. Furthermore, 200 background images were added as background samples to form the image training dataset and a U-Net backbone was used to improve the MaskIoU head of Mask Scoring R–CNN model. This method can improve the efficiency of feature utilisation and promote the reuse of features through the concatenation of feature maps in the process of encoding and decoding. The performance of the MASU R–CNN model was verified by 100 testing images. With ResNet-101 FPN adopted as the feature extraction backbone, the precision of MASU R–CNN reached 96.43%, recall 95.37%, F1 score 95.90%, mean average precision (mAP) 0.594, and mean intersection over union (mIoU) 91.55%. The segmentation results of MASU R–CNN model outperformed those of the other state-of-the-art models. Highlights: An improved Mask R–CNN model processed by U-Net method. Realising instance segmentation of apple flowers in three different growth stages. Combining growth characteristics of apple flowers to realise image augmentation. Realising the segmentation of apple flowers under occlusion and overlap conditions.