Developing and using statistical tools to estimate observer effect for ordered class data: The case of the IBP (Index of Biodiversity Potential). (March 2020)

• Record Type: Journal Article
• Authors: Gosselin, Frédéric; Larrieu, Laurent
• Is Part Of: Ecological indicators. Volume 110(2020)
• Publication Date: 2020-03
• Subjects: Environmental assessment -- Data quality -- Observer variation -- Observer bias -- Hierarchical model -- Bayesian analysis; Environmental monitoring -- Periodicals; Environmental management -- Periodicals; Environmental impact analysis -- Periodicals; Environmental risk assessment -- Periodicals; Sustainable development -- Periodicals; 333.71405
• DOI: 10.1016/j.ecolind.2019.105884
• Languages: English
• ISSNs: 1470-160X

Abstract

Highlights: We studied statistically the variation of an ecological indicator among observers. We used seven factors of the IBP (Index of Biodiversity Potential) as a case study. We developed latent variable models to model ordered class data. An expert observer yielded less noisy IBP scores than other observers. Our work shows the interest of fully parametric statistical models. Abstract: Ecological indicators based on measurements made by individual observers often include extra noise associated with the observer herself (observer effect). The literature on this subject has used a variety of analytical tools to account for this variation, but few analyses have used a hierarchical statistical model approach that can account for the variation and bias among observers on both the mean and variance of the data. We used the Index of Biodiversity Potential (IBP), a rapid habitat assessment method widely used in France, in a case study to propose latent variable hierarchical models to assess the impact of observer effect while accounting for other sources of variation. For seven of the ten factors constituting the IBP, we used a sequential model selection procedure to analyze observers' scores, then analyzed the final models. The structure of the best models varied according to the related factor. Our analyses confirm: (i) that the expert reference observer provided lower variability in his observations than did the other observers for all factors except the one related to standing deadwood; (ii) that all but two factors showed at least a moderate level of systematic random observer variations; (iii) that the absence of leaves in winter led to increased variations when tree-species identification was required; and (iv) that evaluation of stand openness was highly variable and should therefore be assessed by an expert. Overall, the data were coherent with our best models from various points of view for which we diagnosed goodness-of-fit. Our work is a further illustration of the interest of adopting fully parametric statistical modelling of observer-based variations.