Reinforcement learning based dual stocker scheduling

Abstract

The stocker system is the widely used auto material handling system (AMHS) in TFT based FPD fabrication facilities(FABs). In the stocker, a transport vehicle called crane delivers a cassette, production unit of display, to processing machine or shelf. In particular, a dual stocker has two cranes on a single rail, which needs avoidance movement to prevent collision resulting in inefficiency. A scheduling method considering avoidance movement to increase the productivity of the dual stocker, but there has been little research about the problem. In this dissertation, we deal with the scheduling problem of the dual stocker system. We defined the dual stocker scheduling problem (DSSP) using the Markov decision process (MDP), solved the problem by dynamic programming and reinforcement learning using a deep neural network in a static environment. Trace form input shape and convolution layer were used to increase the performance of the neural network. Furthermore, we suggested a DP+DQN model maximizing scheduling efficiency in a dynamic environment based on the problem characteristics from the static case. The developed algorithm was applied to a LEGO hardware system and a dual stocker used in the actual display production line. We verified the applicability of the algorithm to the real system by performance comparison. Besides, we generalized the DSSP to a resource-constrained parallel machine scheduling problem with multi-operation jobs(RCPMSP-M) and suggested that the Graph neural network based reinforcement learning method can learn a good policy for the problem. This provides a good starting point to deal with more complex AMHS scheduling problems. The contributions of this dissertation are as follows: (1) We defined the dual stocker scheduling problem with a new approach and suggested algorithms to get an optimal and good approximate solution. (2) We showed the applicability of reinforcement learning and neural network on the logistic system. (3) We showed that the developed algorithm works in a real system by experiments on LEGO and the actual system. (4) We provided a base for interpreting the interactions between vehicles that inevitably occurred in AMHS. (5) We proposed a Graph neural network based reinforcement learning method to find a good scheduling policy in a complex system.