High-throughput instance segmentation and shape restoration of overlapping vegetable seeds based on sim2real method. (15th February 2023)

• Record Type: Journal Article
• Authors: Liang, Ning; Sun, Sashuang; Zhou, Lei; Zhao, Nan; Taha, Mohamed Farag; He, Yong; Qiu, Zhengjun
• Is Part Of: Measurement. Volume 207(2023)
• Publication Date: 2023-02-15
• Subjects: Seed counting -- Seed phenotyping -- Domain randomization -- Deep learning -- Image processing; Weights and measures -- Periodicals; Measurement -- Periodicals; Measurement; Weights and measures; Periodicals; 530.8
• DOI: 10.1016/j.measurement.2022.112414
• Languages: English
• ISSNs: 0263-2241

Abstract

Highlights: Sim2real thought is adopted to generate simulated images and labels automatically. A novel image simulation method is designed based on multi-feature randomization. Overlapping seeds are segmented rapidly by Mask R-CNN model and sim2real thought. Shape of occluded seeds is restored by Mask R-CNN and sim2real. Abstract: The high-throughput measurement of morphological parameters for seeds is an essential procedure in vegetable phenotyping. This study drew on sim2real thought to design a novel image simulation method based on multi-feature randomization to synthesize simulated images and generate labels automatically. The Mask R-CNN model was optimized based on seed characteristics to improve the segmentation performance of small vegetable seeds. The optimized Mask R-CNN models trained on simulated datasets achieved superior segmentation performance on real-world datasets, and AP@[0.5:0.95] reached more than 0.85. Furthermore, the Mask R-CNN models had a satisfactory performance in the shape restoration of occluded seeds, and AP@[0.5:0.95] were 0.81, 0.79, and 0.80 for eggplant, tomato, and pepper seeds, respectively. The combination of sim2real and Mask R-CNN realized high-throughput instance segmentation and shape restoration of overlapping vegetable seeds, and it could be used as an alternative for vegetable seed phenotyping.