Ladderbot—A conversational agent for human-like online laddering interviews. Issue 171 (March 2023)

• Record Type: Journal Article
• Authors: Rietz, Tim; Maedche, Alexander
• Is Part Of: International journal of human-computer studies. Issue 171(2023)
• Publication Date: 2023-03
• Subjects: Smartphone values -- Laddering -- Means-end approach -- Value-oriented research -- Chatbot; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2022.102969
• Languages: English
• ISSNs: 1071-5819

Abstract

Abstract: In user research, laddering interviews are particularly helpful in eliciting goals and underlying values. However, laddering interviews do not scale due to being time and training intensive. In this study, we propose and evaluate Ladderbot, a text-based conversational agent (CA) capable of facilitating human-like online laddering interviews. Ladderbot uses techniques inspired by face-to-face laddering to engage in an interactive conversation with users. In a between-subject experimental study with 256 participants, we compare Ladderbot against established survey-based laddering approaches in exploring user values for smartphone use. We find that on average, participants participating in CA-based laddering interviews produce twice as many and significantly longer answers. Additionally, we identify the learnability of the CA-based interviews to be significantly higher compared to established survey-based laddering approaches. However, survey-based laddering more reliably produces ladders that end in values, while CA-based laddering trades clear attribute-consequence-value structures to explore negative gains. Therein, besides presenting a new CA-based laddering approach, our study has implications for how user researchers can utilize both survey- and CA-based laddering methods to paint a more complete and comprehensive picture. Highlights: Chatbots may extend the laddering technique by providing an alternative to surveys. Ladderbot encourages participants to provide twice as many and significantly longer answers. Interacting with Ladderbot provides participants with higher learnability. Survey-based laddering may produce more structured results than chatbots.