A generalized energy management framework for hybrid construction vehicles via model-based reinforcement learning. (1st December 2022)

• Record Type: Journal Article
• Authors: Zhang, Wei; Wang, Jixin; Xu, Zhenyu; Shen, Yuying; Gao, Guangzong
• Is Part Of: Energy. Volume 260(2022)
• Publication Date: 2022-12-01
• Subjects: Hybrid construction vehicle -- Energy management -- Model-based learning -- Value function approximation -- Gaussian mixture model; Power resources -- Periodicals; Power (Mechanics) -- Periodicals; Energy consumption -- Periodicals; 333.7905
• DOI: 10.1016/j.energy.2022.124849
• Languages: English
• ISSNs: 0360-5442

Abstract

Abstract: Hybrid construction vehicles (HCVs) have more specific tasks and highly repetitive patterns than on-road vehicles. Consequently, they are more suitable for model-based energy management. However, distinctions between work cycles result in adverse scenarios for generalizing model-based energy management. In this study, we solve this problem by proposing a generalized strategy using a model-based reinforcement learning framework. The generalized design highlights three aspects: 1) long-term stability, 2) self-learning ability, and 3) state transition model reuse. A reward function with a trend term is proposed to avoid the cumulative errors between operation cycles and improve the long-term stability of learning. In addition, Gaussian process regression is leveraged to approximate the value function, thereby reducing the computational load and improving the learning efficiency. To further enhance the reusability of the environmental model, a modelling method based on the Gaussian mixture model is put forward. Finally, a generalized HCV energy management framework that includes offline and online learning is designed, where a pre-learning model and an approximation function are adopted for reuse and dynamic learning. Simulation results demonstrate the superiority of the proposed framework to conventional model-based methods in terms of stability, generality, and adaptability, accompanied by a reduction of 5.9% in fuel consumption. Highlights: A generalized HCV energy management framework via model-based learning is proposed. A novel reward function is designed for better long-term stability of the strategy. Value function approximation method is used to improve the self-learning ability. A method to enhance model reusability based on Gaussian mixture model is developed. Experiments studies show that 5.9% fuel consumption can be saved by this framework.