Can machines talk? Comparison of Eliza with modern dialogue systems. (May 2016)

• Record Type: Journal Article
• Authors: Shah, Huma; Warwick, Kevin; Vallverdú, Jordi; Wu, Defeng
• Is Part Of: Computers in human behavior. Volume 58(2016)
• Page Start: 278
• Page End: 295
• Publication Date: 2016-05
• Subjects: Conversation ability -- Dialogue systems -- Eliza -- Machine -- Talk -- Turing test; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2016.01.004
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: To find if current dialogue systems use the same, psychotherapist questioning technique as Joseph Weizenbaum's 1960 natural language understanding programme, Eliza, the authors carried out an original experiment comparing five successful artificial dialogue systems, Cleverbot, Elbot, Eugene Goostman, JFred and Ultra Hal with an online version of Eliza. More than one hundred male and female participants with 1st or non-1st English language, age range 13–64, interacted with the systems over the Internet scoring each for conversation ability. Developers of the modern conversation systems show they deploy a variety of techniques to initiate and maintain dialogue learning from interactions with humans over the Internet. Statistical significance shows these dialogue systems are an improvement on their predecessor. Embedded on the web affording round-the-clock interaction the nature of artificial dialogue systems is evolving as these systems learn from the way humans converse. The uses of modern Elizas are proven successful as virtual assistants in e-commerce; their conversational basis is already extending into education. What we can say is modern artificial dialogue systems do talk. They are able to participate in conversation in a way their predecessor Eliza could not: they are able to share personal opinions, relay experience of family dramas, be relevant, but also be vague, and mislead just as humans do. Graphical abstract: Highlights: 6 dialogue systems involved in study including online version of Eliza . 116 human judges scored systems' conversation ability. Eugene Goostman and Cleverbot conversational ability scores 2 ½ times over Eliza . Females judges scored dialogue systems higher than males. Judges in age group <25 scored systems higher than age group >25.