Measuring Equity-Promoting Behaviors in Digital Teaching Simulations: A Topic Modeling Approach. (September 2021) - Littenberg-Tobias, Joshua; Borneman, Elizabeth; Reich, Justin
Diversity, equity, and inclusion (DEI) issues are urgent in education. We developed and evaluated a massive open online course ( N = 963) with embedded equity simulations that attempted to equip educators with equity teaching practices. Applying a structural topic model (STM)—a type of natural language processing (NLP)—we examined how participants with different equity attitudes responded in simulations. Over a sequence of four simulations, the simulation behavior of participants with less equitable beliefs converged to be more similar with the simulated behavior of participants with more equitable beliefs ( ES [effect size] = 1.08 SD ). This finding was corroborated by overall changes in equity mindsets ( ES = 0.88 SD ) and changed in self-reported equity-promoting practices ( ES = 0.32 SD ). Digital simulations when combined with NLP offer a compelling approach to both teaching about DEI topics and formatively assessing learner behavior in large-scale learning environments.