Automatic identification of knowledge‐transforming content in argument essays developed from multiple sources. (15th February 2021)

• Record Type: Journal Article
• Authors: Raković, Mladen; Winne, Philip H.; Marzouk, Zahia; Chang, Daniel
• Is Part Of: Journal of computer assisted learning. Volume 37:Number 4(2021)
• Page Start: 903
• Page End: 924
• Publication Date: 2021-02-15
• Subjects: argumentative writing -- evidence -- knowledge telling -- knowledge transforming -- machine learning; Computer-assisted instruction -- Periodicals; 371.334
• DOI: 10.1111/jcal.12531
• Languages: English
• ISSNs: 0266-4909

Abstract

Abstract: Developing knowledge‐transforming skills in writing may help students increase learning by actively building knowledge, regardless of the domain. However, many undergraduate students struggle to transform knowledge when drafting essays based on multiple sources. Writing analytics can be used to scaffold knowledge transforming as writers bring evidence to bear in supporting claims. We investigated how to automatically identify sentences representing knowledge transformation in argumentative essays. A synthesis of cognitive theories of writing and Bloom's typology identified 22 linguistic features to model processes of knowledge transforming in a corpus of 38 undergraduates' essays. Findings indicate undergraduates mostly paraphrase or copy information from multiple sources rather than engage deeply with sources' content. Eight linguistic features were important for discriminating evidential sentences as telling versus transforming source knowledge. We trained a machine learning algorithm that accurately classified nearly three of four evidential sentences as knowledge‐telling or knowledge‐transforming, offering potential for use in future research. Lay Description: What is already known about this topic: Engagement in knowledge transforming in multi‐source writing benefits learning. Post‐secondary writers rarely succeed in knowledge transforming. Computer‐generated formative feedback may promote knowledge‐transforming processes. Computational tools so far developed focused on a composition's rhetorical features only. What this paper adds: We proposed a novel methodology to identify knowledge transforming in argumentative essays. Computational approach involved both rhetorical and content characteristics of evidential sentences. Machine learning algorithm was developed to classify text as knowledge‐transforming versus telling. Eight linguistic features predicted evidential sentences as telling or transforming source knowledge. Implications for practice: There is a clear need to teach post‐secondary students tactics for knowledge transforming. The results can inform the development of writing analytics tool to scaffold knowledge‐transforming revisions.