Effects of simulated observation errors on the performance of species distribution models. Issue 3 (14th November 2018)

• Record Type: Journal Article
• Authors: Fernandes, Rui F.; Scherrer, Daniel; Guisan, Antoine
• Editors: Schröder, Boris
• Is Part Of: Diversity & distributions. Volume 25:Issue 3(2019)
• Page Start: 400
• Page End: 413
• Publication Date: 2018-11-14
• Subjects: artificial data -- AUC -- ecological niche models -- evaluation metric -- habitat suitability models -- Kappa -- model fit -- predictive accuracy -- TSS -- uncertainty; Biodiversity -- Periodicals; Biodiversity conservation -- Periodicals; 577
• DOI: 10.1111/ddi.12868
• Languages: English
• ISSNs: 1366-9516

Abstract

Abstract: Aim: Species distribution information is essential under increasing global changes, and models can be used to acquire such information but they can be affected by different errors/bias. Here, we evaluated the degree to which errors in species data (false presences–absences) affect model predictions and how this is reflected in commonly used evaluation metrics. Location: Western Swiss Alps. Methods: Using 100 virtual species and different sampling methods, we created observation datasets of different sizes (100–400–1, 600) and added increasing levels of errors (creating false positives or negatives; from 0% to 50%). These degraded datasets were used to fit models using generalized linear model, random forest and boosted regression trees. Model fit (ability to reproduce calibration data) and predictive success (ability to predict the true distribution) were measured on probabilistic/binary outcomes using Kappa, TSS, MaxKappa, MaxTSS and Somers'D (rescaled AUC). Results: The interpretation of models' performance depended on the data and metrics used to evaluate them, with conclusions differing whether model fit, or predictive success were measured. Added errors reduced model performance, with effects expectedly decreasing as sample size increased. Model performance was more affected by false positives than by false negatives. Models with different techniques were differently affected by errors: models with high fit presenting lower predictive success (RFs), and vice versa (GLMs). High evaluation metrics could still be obtained with 30% error added, indicating that some metrics (Somers'D) might not be sensitive enough to detect data degradation. Main conclusions: Our findings highlight the need to reconsider the interpretation scale of some commonly used evaluation metrics: Kappa seems more realistic than Somers'D/AUC or TSS. High fits were obtained with high levels of error added, showing that RF overfits the data. When collecting occurrence databases, it is advisory to reduce the rate of false positives (or increase sample sizes) rather than false negatives.