Real-time scheduling for distributed permutation flowshops with dynamic job arrivals using deep reinforcement learning. (October 2022)

• Record Type: Journal Article
• Authors: Yang, Shengluo; Wang, Junyi; Xu, Zhigang
• Is Part Of: Advanced engineering informatics. Volume 54(2022)
• Publication Date: 2022-10
• Subjects: Distributed flowshop scheduling -- Deep reinforcement learning -- Real-time scheduling -- Dynamic job arrivals -- Intelligent scheduling -- Deep Q-network; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2022.101776
• Languages: English
• ISSNs: 1474-0346

Abstract

Highlights: The dynamic distributed permutation flowshop scheduling problem (DPFSP) is studied. The training and execution procedures of scheduling agents are provided. Novel state features, actions, and reward are designed for the scheduling model. Outperforms state-of-the-art meta-heuristics and priority dispatching rules. The deep reinforcement learning can provide real-time scheduling after training. Abstract: Distributed manufacturing plays an important role for large-scale companies to reduce production and transportation costs for globalized orders. However, how to real-timely and properly assign dynamic orders to distributed workshops is a challenging problem. To provide real-time and intelligent decision-making of scheduling for distributed flowshops, we studied the distributed permutation flowshop scheduling problem (DPFSP) with dynamic job arrivals using deep reinforcement learning (DRL). The objective is to minimize the total tardiness cost of all jobs. We provided the training and execution procedures of intelligent scheduling based on DRL for the dynamic DPFSP. In addition, we established a DRL-based scheduling model for distributed flowshops by designing suitable reward function, scheduling actions, and state features. A novel reward function is designed to directly relate to the objective. Various problem-specific dispatching rules are introduced to provide efficient actions for different production states. Furthermore, four efficient DRL algorithms, including deep Q-network (DQN), double DQN (DbDQN), dueling DQN (DlDQN), and advantage actor-critic (A2C), are adapted to train the scheduling agent. The training curves show that the agent learned to generate better solutions effectively and validate that the system design is reasonable. After training, all DRL algorithms outperform traditional meta-heuristics and well-known priority dispatching rules (PDRs) by a large margin in terms of solution quality and computation efficiency. This work shows the effectiveness of DRL for the real-time scheduling of dynamic DPFSP.