Assessing collaborative problem-solving skills among elementary school students. (December 2021)

• Record Type: Journal Article
• Authors: Rojas, Matias; Nussbaum, Miguel; Chiuminatto, Pablo; Guerrero, Orlando; Greiff, Samuel; Krieger, Florian; Van Der Westhuizen, Lindie
• Is Part Of: Computers & education. Volume 175(2021)
• Publication Date: 2021-12
• Subjects: Collaborative problem solving -- Assessment -- PISA 2015 -- Validation -- Design-based research; Education -- Data processing -- Periodicals; Education -- Periodicals; Computers -- Periodicals; Computer-Assisted Instruction -- Periodicals; Éducation -- Informatique -- Périodiques; Electronic journals; 370.285
• DOI: 10.1016/j.compedu.2021.104313
• Languages: English
• ISSNs: 0360-1315

Abstract

Abstract: As 21st century skills have become increasingly important, Collaborative Problem Solving (CPS) is now considered essential in almost all areas of life. Different theoretical frameworks and assessment instruments have emerged for measuring this skill. However, more applied studies on its implementation and evaluation in real-life educational settings are required. In this sense, pre-post experimental designs are essential for identifying new methods for developing collaborative problem-solving skills. To do so, equivalent tests are needed to facilitate consistent score interpretations and reduce the practice effect. In the present study, a Design-Based Research approach is used to design and validate an assessment tool with two equivalent forms based on a framework proposed by the OECD and applied to a collaborative activity. A total of 719 students aged between 10 and 13 years old participated in the different stages of the study. The results show that the proposed instrument effectively measures the problem-solving dimension of collaborative problem-solving skills among students of this age. Moreover, the results from the test were equivalent for both forms and across genders. Finally, there were no significant differences when assessing collaborative problem-solving in human-human groups versus human-agent groups using the proposed instrument. For future work, we recommend including other data sources than just text-based conversations. This would allow us to capture the rich social interactions present in this type of activity. Future work should also consider exploring the extent to which skills could be trained. This could be done in an experimental design assessed using the equivalent forms of the proposed instrument as a pre- and post-test. Doing so would provide a more accurate measure of students' collaborative skills. Highlights: Design-Based Research to develop an assessment based on the OECD Framework. Conditions for collaborative activities were used to develop two equivalent forms. Validating showed that the assessment was measuring the cognitive dimensions of CPS. The study provides evidence in support of using virtual agents during assessments. The assessment, based on the OECD framework, does not measure the social component of CPS.