Multilingual phrase sampling for text entry evaluations. Issue 113 (May 2018) - Franco-Salvador, Marc; Leiva, Luis A.
Highlights: Presents a novel phrase sampling method for multilingual text entry. The method uses knowledge graphs derived from BabelNet. The method has clear benefits over other sampling techniques in several scenarios. Abstract: Text entry evaluations are typically conducted with English-only phrase sets. This calls into question the validity of the results when conducting evaluations with non-native English speakers. Automated phrase sampling methods alleviate this problem, however they are difficult to use in practice and do not take into account language semantics, which is an important attribute to optimize. To achieve this goal, we presentKaps, a phrase sampling method that uses the BabelNet multilingual semantic network as a common knowledge resource, aimed at both standardizing and simplifying the sampling procedure to a great extent. We analyze our method from several perspectives, namely the effect of sampled phrases on user's foreign language proficiency, phrase set memorability and representativeness, and semantic coverage. We also conduct a large-scale evaluation involving native speakers of 10 different languages. Overall, we show that our method is an important step toward and provides unprecedented insight into multilingual text entry evaluations.