FritzBot: A data-driven conversational agent for physical-computing system design. Issue 155 (November 2021) - Chen, Taizhou; Xu, Lantian; Zhu, Kening
Highlights: We construct a lexical circuit-event database for modeling novice users' natural-language behaviors on physical-computing system design and development. For proof of concept, we develop FrtizBot, a BiLSTM-CRF-based system leveraging natural-language interaction for designing and creating Arduino based physical-computing systems. We conduct a user study showing the effectiveness of FritzBot on sup1 porting novice users' physical-computing task through natural-language interaction. Abstract: Creating physical-computing systems, especially selecting correct electronic components, assembling the circuit, and implementing the program, can be challenging for novice users. In this paper, we present FritzBot, a data-driven conversational agent supporting novice users on creating physical-computing systems through natural-language interaction. FritzBot is built upon the structure of a BiLSTM-CRF (bi-directional Long Short-term Memory Network and Conditional Random Field) neural network, as a plug-in for Fritzing. The neural network is trained on a lexical circuit-event database derived from 152 students' reports on their physical-computing course projects. By processing the user's textual description on his/her physical-computing idea, FritzBot can extract the causal relationships between the input and the output events, identify the corresponding electronic components, and generate the Arduino-based circuit and the code along with the step-by-step construction guidelines. Our user study shows that compared to the original Arduino software and the circuit-autocompletion software available in the commercial market, FritzBot significantly shortens the time spent, reduces the perceived workload, and enhances the satisfaction/joy for inexperienced users on designing and prototyping physical-computing systems.