A new joint species distribution model for faster and more accurate inference of species associations from big community data. Issue 11 (12th August 2021)

• Record Type: Journal Article
• Authors: Pichler, Maximilian; Hartig, Florian
• Is Part Of: Methods in ecology and evolution. Volume 12:Issue 11(2021)
• Page Start: 2159
• Page End: 2173
• Publication Date: 2021-08-12
• Subjects: big data -- co‐occurrence -- machine learning -- metacommunity -- regularization -- statistics; Ecology -- Periodicals; Evolution -- Periodicals; 577
• DOI: 10.1111/2041-210X.13687
• Languages: English
• ISSNs: 2041-210X

Abstract

Abstract: Joint species distribution models (JSDMs) explain spatial variation in community composition by contributions of the environment, biotic associations and possibly spatially structured residual covariance. They show great promise as a general analytical framework for community ecology and macroecology, but current JSDMs, even when approximated by latent variables, scale poorly on large datasets, limiting their usefulness for currently emerging big (e.g. metabarcoding and metagenomics) community datasets. Here, we present a novel, more scalable JSDM (sjSDM) that circumvents the need to use latent variables by using a Monte Carlo integration of the joint JSDM likelihood together with flexible elastic net regularization on all model components. We implemented sjSDM in PyTorch, a modern machine learning framework, which allows making use of both CPU and GPU calculations. Using simulated communities with known species–species associations and different number of species and sites, we compare sjSDM with state‐of‐the‐art JSDM implementations to determine computational runtimes and accuracy of the inferred species–species and species–environment associations. We find that sjSDM is orders of magnitude faster than existing JSDM algorithms (even when run on the CPU) and can be scaled to very large datasets. Despite the dramatically improved speed, sjSDM produces more accurate estimates of species association structures than alternative JSDM implementations. We demonstrate the applicability of sjSDM to big community data using eDNA case study with thousands of fungi operational taxonomic units (OTU). Our sjSDM approach makes the analysis of JSDMs to large community datasets with hundreds or thousands of species possible, substantially extending the applicability of JSDMs in ecology. We provide our method in an R package to facilitate its applicability for practical data analysis.