Circular specifications and "predicting" with information from the future: Errors in the empirical SAOM–TERGM comparison of Leifeld & Cranmer. (10th March 2022)

• Record Type: Journal Article
• Authors: Block, Per; Hollway, James; Stadtfeld, Christoph; Koskinen, Johan; Snijders, Tom
• Is Part Of: Network science. Volume 10:Number 1(2022)
• Page Start: 3
• Page End: 14
• Publication Date: 2022-03-10
• Subjects: SAOM -- ERGM -- TERGM -- statistical modeling -- dynamic networks -- inferential network analysis -- longitudinal networks; Social networks -- Research -- Periodicals; System analysis -- Periodicals; System theory -- Periodicals; Computer science -- Periodicals; 003.72
• DOI: 10.1017/nws.2022.6
• Languages: English
• ISSNs: 2050-1242

Abstract

Abstract: We review the empirical comparison of Stochastic Actor-oriented Models (SAOMs) and Temporal Exponential Random Graph Models (TERGMs) by Leifeld & Cranmer in this journal [Network Science 7(1):20–51, 2019]. When specifying their TERGM, they use exogenous nodal attributes calculated from the outcome networks' observed degrees instead of endogenous ERGM equivalents of structural effects as used in the SAOM. This turns the modeled endogeneity into circularity and obtained results are tautological. In consequence, their out-of-sample predictions using TERGMs are based on out-of-sample information and thereby predict the future using observations from the future. Thus, their analysis rests on erroneous model specifications that invalidate the article's conclusions. Finally, beyond these specific points, we argue that their evaluation metric—tie-level predictive accuracy—is unsuited for the task of comparing model performance.