Interpreting log data through the lens of learning design: Second-order predictors and their relations with learning outcomes in flipped classrooms. (July 2021)

• Record Type: Journal Article
• Authors: Wang, Feng Hsu
• Is Part Of: Computers & education. Volume 168(2021)
• Publication Date: 2021-07
• Subjects: Flipped classroom -- Learning management systems -- Online behavioral engagement -- Exploratory factor analysis -- Multiple linear regression; Education -- Data processing -- Periodicals; Education -- Periodicals; Computers -- Periodicals; Computer-Assisted Instruction -- Periodicals; Éducation -- Informatique -- Périodiques; Electronic journals; 370.285
• DOI: 10.1016/j.compedu.2021.104209
• Languages: English
• ISSNs: 0360-1315

Abstract

Abstract: Flipped classrooms supported by learning management systems (LMS) have been widely adopted by educational institutions. However, earlier studies have found problems with interpreting LMS log data to understand student approaches to learning within the context of a learning design. This study investigates whether it is possible to use LMS log data as a proxy to understand students' learning strategies over different periods of time in the flipped-classroom context. A total of 135 sophomores from two classes of a flipped programming course participated in this study. Exploratory factor analysis is first conducted on the log data to synthesize second-order predictors based on the total-effort model. Then, we investigate the extent to which these second-order predictors relate to students' learning outcomes over time. Four types of learning outcomes are considered, including a quiz, a midterm exam, a final exam and the final grade. For each type of learning outcome, multiple linear regression is used to construct a weekly prediction model from these predictors. Adjusted R-squared and RMSE (Root Mean Square Error) are the metrics used to compare the models. The results show that consistent second-order predictors can be derived from log data, implying that students' clicking events in LMS could manifest students' learning strategies understandable in the design context of a flipped classroom. Furthermore, compared with the first-order models, most of the models constructed using the second-order predictors have higher predictive performance, although with lower data fitness. In addition, the predictive performance of the models with MSLQ (Motivated Strategies for Learning Questionnaire) indicators and past assessment data are also examined. It is found that MSQL variables have a positive but short-termed effect on the models' predictive ability, while past assessment data greatly improve the models of all types of learning outcomes. Theoretical contributions and implications of the proposed approach for practice, research and future research are discussed. Highlights: A novel approach to interpreting log data within the design context of FC. Second-order predictors better predict learning performance over time. MSLQ variables only improve prediction for short-term performance. No evidence supports in-class problem solving related to short-term performance. Evidence supports in-class problem solving related to long-term performance.