Reinforcement learning for optimal feedback control : a Lyapunov-based approach /: a Lyapunov-based approach. (2018)

• Record Type: Book
• Further Information: Note: Rushikesh Kamalapurkar, Patrick Walters, Joel Rosenfeld, Warren Dixon.
• Authors: Kamalapurkar, Rushikesh; Walters, Patrick; Rosenfeld, Joel; Dixon, Warren E, 1972-
• Contents: Intro; Preface; Contents; Symbols; 1 Optimal Control; 1.1 Introduction; 1.2 Notation; 1.3 The Bolza Problem; 1.4 Dynamic Programming; 1.4.1 Necessary Conditions for Optimality; 1.4.2 Sufficient Conditions for Optimality; 1.5 The Unconstrained Affine-Quadratic Regulator; 1.6 Input Constraints; 1.7 Connections with Pontryagin's Maximum Principle; 1.8 Further Reading; 1.8.1 Numerical Methods; 1.8.2 Differential Games and Equilibrium Solutions; 1.8.3 Viscosity Solutions and State Constraints; References; 2 Approximate Dynamic Programming; 2.1 Introduction 2.2 Exact Dynamic Programming in Continuous Time and Space; 2.2.1 Exact Policy Iteration: Differential and Integral Methods; 2.2.2 Value Iteration and Associated Challenges; 2.3 Approximate Dynamic Programming in Continuous Time and Space; 2.3.1 Some Remarks on Function Approximation; 2.3.2 Approximate Policy Iteration; 2.3.3 Development of Actor-Critic Methods; 2.3.4 Actor-Critic Methods in Continuous Time and Space; 2.4 Optimal Control and Lyapunov Stability; 2.5 Differential Online Approximate Optimal Control; 2.5.1 Reinforcement Learning-Based Online Implementation 2.5.2 Linear-in-the-Parameters Approximation of the Value Function; 2.6 Uncertainties in System Dynamics; 2.7 Persistence of Excitation and Parameter Convergence; 2.8 Further Reading and Historical Remarks; References; 3 Excitation-Based Online Approximate Optimal Control; 3.1 Introduction; 3.2 Online Optimal Regulation; 3.2.1 Identifier Design; 3.2.2 Least-Squares Update for the Critic; 3.2.3 Gradient Update for the Actor; 3.2.4 Convergence and Stability Analysis; 3.2.5 Simulation; 3.3 Extension to Trajectory Tracking; 3.3.1 Formulation of a Time-Invariant Optimal Control Problem 3.3.2 Approximate Optimal Solution; 3.3.3 Stability Analysis; 3.3.4 Simulation; 3.4 N-Player Nonzero-Sum Differential Games; 3.4.1 Problem Formulation; 3.4.2 Hamilton-Jacobi Approximation Via Actor-Critic-Identifier; 3.4.3 System Identifier; 3.4.4 Actor-Critic Design; 3.4.5 Stability Analysis; 3.4.6 Simulations; 3.5 Background and Further Reading; References; 4 Model-Based Reinforcement Learning for Approximate Optimal Control; 4.1 Introduction; 4.2 Model-Based Reinforcement Learning; 4.3 Online Approximate Regulation; 4.3.1 System Identification; 4.3.2 Value Function Approximation 4.3.3 Simulation of Experience Via Bellman Error Extrapolation; 4.3.4 Stability Analysis; 4.3.5 Simulation; 4.4 Extension to Trajectory Tracking; 4.4.1 Problem Formulation and Exact Solution; 4.4.2 Bellman Error; 4.4.3 System Identification; 4.4.4 Value Function Approximation; 4.4.5 Simulation of Experience; 4.4.6 Stability Analysis; 4.4.7 Simulation; 4.5 N-Player Nonzero-Sum Differential Games; 4.5.1 System Identification; 4.5.2 Model-Based Reinforcement Learning; 4.5.3 Stability Analysis; 4.5.4 Simulation; 4.6 Background and Further Reading; References; 5 Differential Graphical Games
• Publisher Details: Cham, Switzerland : Springer
• Publication Date: 2018
• Extent: 1 online resource (xvi, 293 pages), illustrations
• Subjects: 006.3/1; Engineering; Reinforcement learning; Feedback control systems; COMPUTERS / General; Feedback control systems; Reinforcement learning; Mathematics -- Calculus; Science -- System Theory; Technology & Engineering -- Telecommunications; Calculus of variations; Cybernetics & systems theory; Communications engineering / telecommunications; Mathematical optimization; Systems theory; Telecommunication; Technology & Engineering -- Automation; Automatic control engineering; Electronic books
• Languages: English
• ISBNs: 9783319783840
• Related ISBNs: 331978384X; 9783319783833; 3319783831
• Notes: Note: Includes bibliographical references and index.; Note: Online resource; title from PDF title page (SpringerLink, viewed May 15, 2018).
• Access Rights: Legal Deposit; Only available on premises controlled by the deposit library and to one user at any one time; The Legal Deposit Libraries (Non-Print Works) Regulations (UK).
• Access Usage: Restricted: Printing from this resource is governed by The Legal Deposit Libraries (Non-Print Works) Regulations (UK) and UK copyright law currently in force.