A crowdsourcing semi-automatic image segmentation platform for cell biology. (March 2021)

• Record Type: Journal Article
• Authors: Bafti, Saber Mirzaee; Ang, Chee Siang; Hossain, Md Moinul; Marcelli, Gianluca; Alemany-Fornes, Marc; Tsaousis, Anastasios D.
• Is Part Of: Computers in biology and medicine. Volume 130(2021)
• Publication Date: 2021-03
• Subjects: Semi-auto segmentation -- Object detection -- Computational biology -- Crowdsourcing -- Image annotation -- Instance segmentation; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2020.104204
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: State-of-the-art computer-vision algorithms rely on big and accurately annotated data, which are expensive, laborious and time-consuming to generate. This task is even more challenging when it comes to microbiological images, because they require specialized expertise for accurate annotation. Previous studies show that crowdsourcing and assistive-annotation tools are two potential solutions to address this challenge. In this work, we have developed a web-based platform to enable crowdsourcing annotation of image data; the platform is powered by a semi-automated assistive tool to support non-expert annotators to improve the annotation efficiency. The behavior of annotators with and without the assistive tool is analyzed, using biological images of different complexity. More specifically, non-experts have been asked to use the platform to annotate microbiological images of gut parasites, which are compared with annotations by experts. A quantitative evaluation is carried out on the results, confirming that the assistive tools can noticeably decrease the non-expert annotation's cost (time, click, interaction, etc.) while preserving or even improving the annotation's quality. The annotation quality of non-experts has been investigated using IoU (intersection over union), precision and recall; based on this analysis we propose some ideas on how to better design similar crowdsourcing and assistive platforms. Highlights: Our assistive tool enables non-expert annotators to perform annotation of microbiological images accurately and quickly. Our study sheds some light on important behavioral features of non-expert annotators. Our findings can help inform the design of future annotation systems. Annotation quality is not found to be strongly correlated with either annotation time or the number of regions. Other than time reduction, annotation platforms should encourage annotators toward more efficient annotations.