Matrix methods for stochastic dynamic programming in ecology and evolutionary biology. Issue 11 (26th September 2019)

• Record Type: Journal Article
• Authors: Reimer, Jody R.; Mangel, Marc; Derocher, Andrew E.; Lewis, Mark A.
• Editors: Ergon, Torbjørn
• Is Part Of: Methods in ecology and evolution. Volume 10:Issue 11(2019)
• Page Start: 1952
• Page End: 1961
• Publication Date: 2019-09-26
• Subjects: backwards induction -- Markov chain -- Markov decision process -- optimality models -- stationary decisions -- stochastic dynamic programming -- value iteration; Ecology -- Periodicals; Evolution -- Periodicals; 577
• DOI: 10.1111/2041-210X.13291
• Languages: English
• ISSNs: 2041-210X

Abstract

Abstract: Organisms are constantly making tradeoffs. These tradeoffs may be behavioural (e.g. whether to focus on foraging or predator avoidance) or physiological (e.g. whether to allocate energy to reproduction or growth). Similarly, wildlife and fishery managers must make tradeoffs while striving for conservation or economic goals (e.g. costs vs. rewards). Stochastic dynamic programming (SDP) provides a powerful and flexible framework within which to explore these tradeoffs. A rich body of mathematical results on SDP exist but have received little attention in ecology and evolution. Using directed graphs – an intuitive visual model representation – we reformulated SDP models into matrix form. We synthesized relevant existing theoretical results which we then applied to two canonical SDP models in ecology and evolution. We applied these matrix methods to a simple illustrative patch choice example and an existing SDP model of parasitoid wasp behaviour. The proposed analytical matrix methods provide the same results as standard numerical methods as well as additional insights into the nature and quantity of other, nearly optimal, strategies, which we may also expect to observe in nature. The mathematical results highlighted in this work also explain qualitative aspects of model convergence. An added benefit of the proposed matrix notation is the resulting ease of implementation of Markov chain analysis (an exact solution for the realized states of an individual) rather than Monte Carlo simulations (the standard, approximate method). It also provides an independent validation method for other numerical methods, even in applications focused on short‐term, non‐stationary dynamics. These methods are useful for obtaining, interpreting, and further analysing model convergence to the optimal time‐independent (i.e. stationary) decisions predicted by an SDP model. SDP is a powerful tool both for theoretical and applied ecology, and an understanding of the mathematical structure underlying SDP models can increase our ability to apply and interpret these models.