Gradient-Bounded Dynamic Programming with Submodular and Concave extensible Value Functions. Issue 2 (2020)

• Record Type: Journal Article
• Authors: Lebedev, Denis; Goulart, Paul; Margellos, Kostas
• Is Part Of: IFAC-PapersOnLine. Volume 53:Issue 2(2020)
• Page Start: 6825
• Page End: 6830
• Publication Date: 2020
• Subjects: Dual dynamic programming -- Function approximation -- Real-time operations in transportation; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2020.12.337
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: We consider dynamic programming problems with finite, discrete-time horizons and prohibitively high-dimensional, discrete state-spaces for direct computation of the value function from the Bellman equation. For the case that the value function of the dynamic program is concave extensible and submodular in its state-space, we present a new algorithm that computes deterministic upper and stochastic lower bounds of the value function similar to dual dynamic programming. We then show that the proposed algorithm terminates after a fnite number of iterations. Finally, we demonstrate the efficacy of our approach on a high-dimensional numerical example from delivery slot pricing in attended home delivery.