Action space designs for stochastic production scheduling with sequence-dependent setup times

We investigate flexible shop scheduling problems with sequence-dependent setup times. In many practical environments, setup times as well as processing times are stochastic, requiring solution approaches that are able to react dynamically to un- certainty. The problem can be expressed as a dynamic programming problem in which a new decision epoch is reached whenever a machine becomes idle. Accordingly, we de- fine a Markov decision process (MDP) formulation, which serves as a generic basis for real-time scheduling. As policy approximations, we suggest multiple methods based on (Deep) Reinforcement Learning (RL) and priority rules developed by Ge- netic Programming (GP). The design of the action space defines the solution space and thus influences the quality of the policy approximation. Action spaces are cre- ated to address the additional complexity imposed by specific characteristic of the problem, such as sequence dependent setups. We evaluate our different action space designs regarding their suitability for solution methods, and their ability to generalize.      «

We investigate flexible shop scheduling problems with sequence-dependent setup times. In many practical environments, setup times as well as processing times are stochastic, requiring solution approaches that are able to react dynamically to un- certainty. The problem can be expressed as a dynamic programming problem in which a new decision epoch is reached whenever a machine becomes idle. Accordingly, we de- fine a Markov decision process (MDP) formulation, which serves as a generic b...     »