Crowdsourced real-time captioning of sign language by deaf and hard-of-hearing people. Issue 1 (3rd April 2017)

• Record Type: Journal Article
• Authors: Shiraishi, Yuhki; Zhang, Jianwei; Wakatsuki, Daisuke; Kumai, Katsumi; Morishima, Atsuyuki
• Is Part Of: International journal of pervasive computing and communications. Volume 13:Issue 1(2017)
• Page Start: 2
• Page End: 25
• Publication Date: 2017-04-03
• Subjects: Crowdsourcing -- Deaf and hard-of-hearing people -- Information support -- Real-time captioning -- Sign language; Ubiquitous computing -- Periodicals; Mobile computing -- Periodicals; Computer network protocols -- Periodicals; Computer network architectures -- Periodicals; Application software -- Development -- Periodicals; 004.6
• DOI: 10.1108/IJPCC-02-2017-0014
• Languages: English
• ISSNs: 1742-7371

Abstract

Abstract : Purpose: The purpose of this paper is to explore the issues on how to achieve crowdsourced real-time captioning of sign language by deaf and hard-of-hearing (DHH) people, such that how a system structure should be designed, how a continuous task of sign language captioning should be divided into microtasks and how many DHH people are required to maintain a high-quality real-time captioning. Design/methodology/approach: The authors first propose a system structure, including the new design of worker roles, task division and task assignment. Then, based on an implemented prototype, the authors analyze the necessary setting for achieving a crowdsourced real-time captioning of sign language, test the feasibility of the proposed system and explore its robustness and improvability through four experiments. Findings: The results of Experiment 1 have revealed the optimal method for task division, the necessary minimum number of groups and the necessary minimum number of workers in a group. The results of Experiment 2 have verified the feasibility of the crowdsourced real-time captioning of sign language by DHH people. The results of Experiment 3 and Experiment 4 have shown the robustness and improvability of the captioning system. Originality/value: Although some crowdsourcing-based systems have been developed for the captioning of voice to text, the authors intend to resolve the issues on the captioning of sign language to text, for which the existing approaches do not work well due to the unique properties of sign language. Moreover, DHH people are generally considered as the ones who receive support from others, but our proposal helps them become the ones who offer support to others.