Rich data, poor information? Teachers' perceptions of mean differences in graphical feedback from statewide tests. (April 2023)

• Record Type: Journal Article
• Authors: Merk, Samuel; Groß Ophoff, Jana; Kelava, Augustin
• Is Part Of: Learning and instruction. Volume 84(2023)
• Publication Date: 2023-04
• Subjects: Standardized tests -- Data-based decision making -- Data literacy -- Graph comprehension -- Teacher education; Learning -- Periodicals; Teaching -- Periodicals; Apprentissage -- Périodiques; Enseignement -- Périodiques; Learning; Teaching; Periodicals; Electronic journals; 370.1
• DOI: 10.1016/j.learninstruc.2022.101717
• Languages: English
• ISSNs: 0959-4752

Abstract

Abstract: Data-based decision making in schools aims to address students' needs and improve their performance by leveraging the information contained in data. However, to learn from data, teachers are required to transform them into actionable information. In two studies, we focused on how teachers can learn about the strengths and weaknesses of their class performance from statewide test data by investigating how sensitively, efficiently, and accurately they are able to judge the magnitude of mean differences. The results indicate that teachers can judge mean differences with very high accuracy (τ = .85) but only consider moderate and large effects to be meaningful, indicating low sensitivity. By systematically varying different graph types, we also found that participants quickly increased their efficiency in terms of dwell time when repeatedly confronted with a specific graph type and were most sensitive to the use of stacked or clustered bar graphs. Highlights: Data-based decision making requires an understanding of the data. Teachers can judge mean differences with very high accuracy (τ = 0.85). But teachers only consider moderate and large effects to be meaningful.