Toward a mixed-initiative QA system: from studying predictors in Stack Exchange to building a mixed-initiative tool. Issue 99 (March 2017)

• Record Type: Journal Article
• Authors: Convertino, Gregorio; Zancanaro, Massimo; Piccardi, Tiziano; Ortega, Felipe
• Is Part Of: International journal of human-computer studies. Issue 99(2017)
• Page Start: 1
• Page End: 20
• Publication Date: 2017-03
• Subjects: Question Answering sites -- Crowdsourcing -- Mixed-initiative tools -- Customer Support; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2016.10.008
• Languages: English
• ISSNs: 1071-5819

Abstract

Abstract: This article envisions a new customer support solution that merges the efficiency of crowd-based Question and Answer (QA) sites with the effectiveness of traditional customer care services. QA sites use crowdsourcing to solve problems in a very efficient way and they represent a new approach that can compete with traditional customer support services. Despite the remarkable efficiency of popular QA sites, if a question is not solved almost immediately, the chances are that it will not be solved soon or perhaps ever. This article provides evidence of a consistent Dark Side, a group of questions that remain unsatisfied or are satisfied very late, in eight popular QA sites on Stack Exchange. About 25–30% of all the questions in these sites fall into this Dark Side group. The findings show that predicting if a question will end up in the Dark Side is feasible, although with some approximation, without relying on content features. On the basis of this evidence, the article first presents and tests a model to predict the Dark Side and then presents a proof-of-concept of a mixed-initiative tool that helps a crowd-manager to decide whether an incoming question will be solved by the crowd or it should be redirected to a dedicated operator. Multiple evaluations of the proposed tool are reported. Finally, it concludes with lessons for the design and management of future QA platforms. Highlights: Evidence of a consistent Dark Side in eight popular QA (Question and Answer) sites. A mixed-initiative system for a crowd-manager to handle unanswered questions. Two types of evaluations are reported: a laboratory experiment and a simulation study. A logistic regression model to automatically predict the Dark Side questions.