Exploring the potential of natural language processing to support microgenetic analysis of collaborative learning discussions. (26th August 2019)

• Record Type: Journal Article
• Authors: Sullivan, Florence R.; Keith, P. Kevin
• Is Part Of: British journal of educational technology. Volume 50:Number 6(2019:Nov.)
• Page Start: 3047
• Page End: 3063
• Publication Date: 2019-08-26
• Subjects: Audio-visual education -- Periodicals; 371.33
• DOI: 10.1111/bjet.12875
• Languages: English
• ISSNs: 0007-1013

Abstract

Abstract: In this study, we explore the potential of a natural language processing (NLP) approach to support discourse analysis of in‐situ, small group learning conversations. The theoretical basis of this work derives from Bakhtin's notion of speech genres as bounded by educational robotics activity. Our goal is to leverage computational linguistics methods to advance and improve educational research methods. We used a parts‐of‐speech (POS) tagging program to automatically parse a transcript of spoken dialogue collected from a small group of middle school students involved in solving a robotics challenge. We grammatically parsed the dialogue at the level of the trigram. Then, through a deliberative process, we mapped the POS trigrams to our theoretically derived problem solving in computational environments coding system. Next, we developed a stacked histogram visualization to identify rich interactional segments in the data. Seven segments of the transcript were thus identified for closer analysis. Our NLP‐based approach partially replicated prior findings. Here, we present the theoretical basis for the work, our analytical approach in exploring this NLP‐based method, and our research findings. Practitioner Notes What is already known about this topic Over the last 10 years, several educational research papers indicate that natural language processing (NLP) techniques can be used to help interpret well‐structured, written dialogue, eg, conversations in online class discussions. Two recent papers indicate that NLP techniques can also be used to help interpret well‐structured, spoken dialogue, eg, replies to interview questions and/or comments made during think aloud protocols. Multimodal learning analytic techniques are being used to investigate collaborative learning. These studies use non‐verbal features of data (gaze, gesture, physical actions), prosodic features of verbal data (pitch and tone) and/or turn‐taking and duration of talk per speaker data, as means of predicting group success. None of the MMLA studies attempt semantic analysis of student talk in collaborative settings. What this paper adds A theoretical framework for why and how an automated NLP approach can support discourse analysis research on co‐located, computer‐based, collaborative problem solving interactions. This framework, entitled the Problem Solving in Computational Environment Speech Genre, links children's physical interactions with computational devices to their verbal exchanges and presents a theoretical rationale for the use of NLP methods in educational research. Description of an interdisciplinary method that combines NLP techniques with qualitative coding approaches to support analysis of student collaborative learning with educational robotics. Identification of student learning outcomes derived from the semantic, PSCE Speech Genre and NLP approach. Implications for practice and/or policy Educational researchers will be able to expand upon our findings towards the goal of using computation and automation to support microgenetic analysis of large datasets. Robust microgenetic learning findings will provide curriculum developers, educational technology developers and teachers with guidance on how to construct and or create learning materials and environments. From an interdisciplinary perspective, this research can support more interdisciplinary exploration of conversational dialogues that are ill‐structured, indexical and referential. This research will support the further development of machine learning techniques and neural network models by computational linguists.