Enabling intelligent onboard guidance, navigation, and control using reinforcement learning on near-term flight hardware. (October 2022)

• Record Type: Journal Article
• Authors: Wilson, Callum; Riccardi, Annalisa
• Is Part Of: Acta astronautica. Volume 199(2022)
• Page Start: 374
• Page End: 385
• Publication Date: 2022-10
• Subjects: Intelligent control -- Reinforcement learning -- Edge artificial intelligence; Astronautics -- Periodicals; Outer space -- Exploration -- Periodicals; Astronautics; Periodicals; 629.405
• DOI: 10.1016/j.actaastro.2022.07.013
• Languages: English
• ISSNs: 0094-5765

Abstract

Abstract: Future space missions require technological advances to meet more stringent requirements. Next generation guidance, navigation, and control systems must safely operate autonomously in hazardous and uncertain environments. While these developments often focus on flight software, spacecraft hardware also creates computational limitations for onboard algorithms. Intelligent control methods combine theories from automatic control, artificial intelligence, and operations research to derive control systems capable of handling large uncertainties. While this can be beneficial for spacecraft control, such control systems often require substantial computational power. Recent improvements in single board computers have created physically lighter and less power-intensive processors that are suitable for spaceflight and purpose built for machine learning. In this study, we implement a reinforcement learning based controller on NVIDIA Jetson Nano hardware and apply this controller to a simulated Mars powered descent problem. The proposed approach uses optimal trajectories and guidance laws under nominal environment conditions to initialise a reinforcement learning agent. This agent learns a control policy to cope with environmental uncertainties and updates its control policy online using a novel update mechanism called Extreme Q-Learning Machine. We show that this control system performs well on flight suitable hardware, which demonstrates the potential for intelligent control onboard spacecraft. Graphical abstract: Highlights: Demonstration of an intelligent control algorithm running on hardware suitable for spaceflight. Proposed method combines optimal control demonstrations with reinforcement learning. Q-network updates its weights online using a novel update mechanism called Extreme Q-Learning Machine. Reinforcement learning agent applied to a simulated Mars lander environment.