Further evaluating interobserver reliability and accuracy with and without structured visual‐inspection criteria. (18th May 2021) - Cox, Alison D.; Zonneveld, Kimberley L. M.; Tardi, Laura D.
Abstract: Visual inspection is the primary method of interpreting functional analysis (FA) outcomes, even though it has occasionally been criticized for producing low levels of interobserver agreement. Researchers have addressed this issue by creating structured visual‐inspection criteria to guide visual inspection of FA outcomes (e.g., Hagopian et al., 1997, https://doi.org/10.1901/jaba.1997.30‐313 ; Roane et al., 2013, https://doi.org/10.1002/jaba.13 ). The purpose of the current study was to systematically replicate and extend Study 1 of Roane et al. (2013, https://doi.org/10.1002/jaba.13 ). We did this by evaluating the reliability and accuracy of 15 novice participants' visual inspection of 84 FA graphs with and without the modified visual‐inspection criteria developed by Roane et al. Accuracy was markedly higher when participants used the modified visual‐inspection criteria relative to when they used traditional visual‐inspection strategies, while we observed more modest increases in reliability coefficients. Results are discussed in the context of practical and clinical implications of the modified visual‐inspection criteria and suggestions for future research.