Magnitude-based inference and its application in user research. Issue 88 (April 2016) - van Schaik, Paul; Weston, Matthew
Abstract: Magnitude-based inference offers a theoretically justified and practically useful approach in any behavioural research that involves statistical inference. This approach supports two important types of inference: mechanistic inference and practical inference to support real-world decision-making. Therefore, this approach is especially suitable for user research. We present basic elements of magnitude-based inference and examples of its application in user research as well as its merits. Finally, we discuss other approaches to statistical inference and limitations of magnitude-based inference, and give recommendations on how to use this type of inference in user research. Highlights: Magnitude-based inference is a useful alternative for analysing user-research data. Goal-setting in user research is supported by choosing a smallest important effect. The approach uses the smallest important effect in making an inference. As a consequence, a clear effect is never an artefact of sample size. Practical inference is supported by weighing harm and benefit appropriately.