Deep reinforcement learning for a color-batching resequencing problem. (July 2020)

• Record Type: Journal Article
• Authors: Leng, Jinling; Jin, Chun; Vogl, Alexander; Liu, Huiyu
• Is Part Of: Journal of manufacturing systems. Volume 56(2020)
• Page Start: 175
• Page End: 187
• Publication Date: 2020-07
• Subjects: Deep reinforcement learning -- Color-Batching problem -- Virtual car resequencing -- Production control -- Automotive industry; Manufacturing processes -- Periodicals; Production engineering -- Data processing -- Periodicals; Robots, Industrial -- Periodicals; Production, Technique de la -- Informatique -- Périodiques; Robots industriels -- Périodiques; Electronic journals; 670.42
• DOI: 10.1016/j.jmsy.2020.06.001
• Languages: English
• ISSNs: 0278-6125

Abstract

Highlights: Proposed a color-histogram model to describe the color-batching resequencing problem (CRP) as a Markov Decision Process. The number of possible actions was reduced to fit deep reinforcement learning (DRL) approach. Proposed a Deep Q-Network (DQN)-based DRL algorithm can learn and adapt to the practical CRP parameters. The proposed algorithm outperformed benchmarks in a practical-scale experiment in terms of cost of paint color changeover. Abstract: In automotive paint shops, changes of colors between consecutive production orders cause costs for cleaning the painting robots. It is a significant task to re-sequence orders and group orders with identical color as a color batch to minimize the color changeover costs. In this paper, a Color-batching Resequencing Problem (CRP) with mix bank buffer systems is considered. We propose a Color-Histogram (CH) model to describe the CRP as a Markov decision process and a Deep Q-Network (DQN) algorithm to solve the CRP integrated with the virtual car resequencing technique. The CH model significantly reduces the number of possible actions of the DQN agent, so that the DQN algorithm can be applied to the CRP at a practical scale. A DQN agent is trained in a deep reinforcement learning environment to minimize the costs of color changeovers for the CRP. Two experiments with different assumptions on the order attribute distributions and cost metrics were conducted and evaluated. Experimental results show that the proposed approach outperformed conventional algorithms under both conditions. The proposed agent can run in real time on a regular personal computer with a GPU. Hence, the proposed approach can be readily applied in the production control of automotive paint shops to resolve order-resequencing problems.