Assessing individual contributions to Collaborative Problem Solving: A network analysis approach. (March 2020)

• Record Type: Journal Article
• Authors: Swiecki, Zachari; Ruis, A.R.; Farrell, Cayley; Shaffer, David Williamson
• Is Part Of: Computers in human behavior. Volume 104(2020)
• Publication Date: 2020-03
• Subjects: Collaborative Problem Solving -- Epistemic Network Analysis -- Coding and counting -- Assessment -- Measurement; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2019.01.009
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Collaborative Problem Solving (CPS) is an interactive, interdependent, and temporal process. However, current methods for measuring the CPS processes of individuals, such as coding and counting, treat these processes as sets of isolated and independent events. In contrast, Epistemic Network Analysis (ENA) models how the contributions of a given individual relate to the contributions of others. This article examines the communications of air defense warfare teams from an experiment comparing two different computer-based decision support systems, using this data to ask whether ENA provides a more ecologically valid quantitative model of CPS than coding and counting. Qualitative analysis showed that commanders using one system asked questions to understand the tactical situation, while commanders using an experimental system focused more on actions in response to the tactical situation. Neither of the coding and counting approaches we tested corroborated these findings with statistically significant results. In contrast, ENA created models of the individual contributions of commanders that (a) showed statistical differences between commanders using the two systems to corroborate the qualitative analysis, and (b) revealed differences in individual performance. This suggests that ENA is a more powerful tool for CPS assessment than coding and counting approaches. Highlights: Coding and counting models CPS processes of individuals as isolated and independent. ENA models CPS processes of individuals in relation to the processes of others. ENA found statistically significant differences between individuals in team settings. ENA found significant differences where coding and counting did not. Significant ENA differences were aligned with qualitative results.