Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning

Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning Junyoung Park Junyoung Park Jaehyeong Chun Sang Hun Kim Youngkook Kim Jinkyoo Park Jinkyoo Park

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning 2021 International Journal of Production Research 10.1080/00207543.2020.1870013 park2021-learning-to-schedule-job-shop-prob park2021-learning-to-schedule-job-shop-prob.txt, the project's staged copy 2b5385facbb28a4f7b21fa7e0e7cb32194f57c76 open-access machine-learning

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. job-shop scheduling Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. mechanisms Because the embedding layer works through pairwise interaction functions that are shared across all node pairs rather than tied to a fixed graph size, the same trained network can be run on graphs with different node counts and edge patterns, which is what carries the policy to unseen instance sizes Quotation withheld because this record is not cleared for redistribution. mechanisms Training on a graph representation forces the policy to key on the structural regularities common to all job shops instead of memorising one instance, which is why it transfers with no further training Quotation withheld because this record is not cleared for redistribution. mechanisms The learned policy beats fixed rules because it re-weights which state features matter at each decision, whereas a dispatching rule always consults the same predetermined features Quotation withheld because this record is not cleared for redistribution. mechanisms The dense waiting-job reward is used instead of makespan because a reward delivered only at episode end leaves the algorithm unable to assign credit to individual dispatching actions Quotation withheld because this record is not cleared for redistribution. Computation time for node embeddings grows rapidly (quadratically in the number of jobs) with instance size. what this limits: the practical scalability claim, not the solution quality: THE COMPUTATIONAL TIME TO COMPUTE THE NODE EMBEDDINGS RAPIDLY INCREASES with instance size Quotation withheld because this record is not cleared for redistribution. The MDP formulation omits practical shop-floor constraints such as batching, waiting-time regulations and flexible setups. what this limits: any shop-floor deployment reading: THE CURRENT FORMULATION DOES NOT CONSIDER MORE PRACTICAL CONSTRAINTS SUCH AS BATCH PROCESSES AND WAITING TIME REGULATIONS AND FLEXIBLE SHOP SETUP Quotation withheld because this record is not cleared for redistribution. The distribution used to generate training and test instances does not cover real-world job shops. what this limits: any claim of applicability to real job shops: THE DISTRIBUTION P0(g0) USED TO GENERATE THE TRAINING AND TEST JSSP INSTANCES IS NOT GENERAL ENOUGH TO COVER REAL-WORLD JSSPs Quotation withheld because this record is not cleared for redistribution. The comparison with prior RL schedulers could only test limited generalisation, because those baselines require train and test instances of the same size. what this limits: the comparison against prior RL schedulers: ONLY A LIMITED GENERALIZATION TEST COULD BE CONDUCTED FOR THE CASE WHERE THE CONJUNCTIVE CONSTRAINTS AND PROCESSING TIMES ARE DIFFERENT FROM THE TRAINING AND TEST DATA SETS Quotation withheld because this record is not cleared for redistribution. The learned scheduler is slower at inference than simple priority dispatching rules. what this limits: any deployment where scheduling latency matters: THE PROPOSED GNN-SCHEDULER REQUIRES MORE COMPUTATIONAL TIME THAN PDRs Quotation withheld because this record is not cleared for redistribution. The headline comparison is scoped to instances drawn from the training distribution, and the comparison against the other learned schedulers is scoped by a restriction on testing instances. what this limits: the claim that the GNN scheduler beats dispatching rules and other RL schedulers in general: the PDR comparison is on instances from the training distribution, and the RL comparison carries a stated restriction on testing instances Quotation withheld because this record is not cleared for redistribution. The method's cost grows with problem size by the authors' own analysis, which is the quantity a scheduling practitioner would care about. what this limits: applying this scheduler to large shops: the authors state embedding time grows in proportion to n2 + nm in the numbers of jobs and machines Quotation withheld because this record is not cleared for redistribution. The reference against which solution quality is judged is another solver's output, not a proven optimum supplied independently. what this limits: the optimality gaps reported for the validation set: the reference minimum makespan is computed by the Google OR tool rather than supplied independently Quotation withheld because this record is not cleared for redistribution. Training stops on a convergence criterion measured on the validation set, so the reported model is the one that stopped improving on that set. what this limits: the trained policy as a fixed artefact: training was terminated when the validation-set makespan converged, so the model is defined by that stopping rule Quotation withheld because this record is not cleared for redistribution. The architecture search behind the reported hyperparameters is reported only as a negative result, with no numbers attached. what this limits: the reported architecture as tuned: deeper and thicker networks were tested and are reported not to have improved performance, without the comparison being shown Quotation withheld because this record is not cleared for redistribution. The size-agnostic transferability of graph neural networks, demonstrated in robot control, wind power estimation and weather prediction, is imported into production scheduling to make a scheduler that works on unseen shop sizes Quotation withheld because this record is not cleared for redistribution. Proximal Policy Optimization, a policy-gradient algorithm from the RL literature, is adopted to train the representation and dispatching modules of a job-shop scheduler end to end Quotation withheld because this record is not cleared for redistribution. the 6x6 instances were dropped from the performance evaluation because only one benchmark problem exists at that size eliminates: that a mean over problem sizes is dominated by a size represented by a single instance · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. job shop scheduling problem job shop scheduling problem Quotation withheld because this record is not cleared for redistribution. MDP MDP Quotation withheld because this record is not cleared for redistribution. makespan makespan Quotation withheld because this record is not cleared for redistribution. priority dispatching rules priority dispatching rules Quotation withheld because this record is not cleared for redistribution. generalization - AS THE ABSTRACT CLAIMS IT: a superb capability, asserted without a scope Quotation withheld because this record is not cleared for redistribution. generalization - AS THE CONCLUSION SCOPES IT: trained on only small JSSPs and producing high-quality schedules for large ones, with the PDR comparison run on the training distribution Quotation withheld because this record is not cleared for redistribution. scheduling quality - operationalised entirely as makespan: the maximum completion time among jobs, which is the single quantity the reward function is built to minimise Quotation withheld because this record is not cleared for redistribution. builds-on → Proximal Policy Optimization (Schulman et al., cited as [37]/[38]) Quotation withheld because this record is not cleared for redistribution. extends → MARL [10] and MDQN [24] RL-based JSSP schedulers Quotation withheld because this record is not cleared for redistribution. job shop scheduling problem —operationalised-as→ MDP job shop scheduling problem MDP strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span writes 'JSSP'; endpoint abbreviated. Quotation withheld because this record is not cleared for redistribution. regression (polynomial curve fitting) regression (polynomial curve fitting) Quotation withheld because this record is not cleared for redistribution. coefficient of determination coefficient of determination Quotation withheld because this record is not cleared for redistribution. standard deviation standard deviation Quotation withheld because this record is not cleared for redistribution. error bars (one standard deviation) error bars (one standard deviation) Quotation withheld because this record is not cleared for redistribution. mean mean Quotation withheld because this record is not cleared for redistribution. On test instances the trained GNN scheduler reached 14.7% average relative error. Quotation withheld because this record is not cleared for redistribution. On published benchmark sets the GNN scheduler gave average relative errors from 12.6% to 20.9%. Quotation withheld because this record is not cleared for redistribution. A scheduler trained only on small randomly generated instances transfers to much larger instances, since the gap between the generically trained GNN0 and the size-matched GNN1-3 was small. Quotation withheld because this record is not cleared for redistribution. The GNN scheduler reliably produced high-quality solutions across instance types. Quotation withheld because this record is not cleared for redistribution. Computation time grows roughly with the square of the number of jobs (quadratic fit R2 = 0.998). Quotation withheld because this record is not cleared for redistribution. Stacking more than three embedding layers bought no measurable scheduling improvement while costing extra computation, so the message-passing depth was fixed at three. Quotation withheld because this record is not cleared for redistribution. The GNN scheduler needs more computation time than the priority dispatching rules. Quotation withheld because this record is not cleared for redistribution. Fewer than 20 episodes per parameter update left the state-value estimate with large variance, while more than 20 added no performance, fixing the batch size at 20 episodes. Quotation withheld because this record is not cleared for redistribution. job shop scheduling problem (JSSP) job shop scheduling problem (JSSP) Quotation withheld because this record is not cleared for redistribution. priority dispatching rule priority dispatching rule Quotation withheld because this record is not cleared for redistribution. graph state for the JSSP graph state for the JSSP Quotation withheld because this record is not cleared for redistribution. non-trivial state non-trivial state Quotation withheld because this record is not cleared for redistribution. primitive transition primitive transition Quotation withheld because this record is not cleared for redistribution. graph neural network (GNN) graph neural network (GNN) Quotation withheld because this record is not cleared for redistribution. makespan makespan Quotation withheld because this record is not cleared for redistribution. number of jobs number of jobs Quotation withheld because this record is not cleared for redistribution. number of machines number of machines Quotation withheld because this record is not cleared for redistribution. relative scheduling error relative scheduling error Quotation withheld because this record is not cleared for redistribution. computation time computation time Quotation withheld because this record is not cleared for redistribution. additional training additional training Quotation withheld because this record is not cleared for redistribution. discount factor discount factor Quotation withheld because this record is not cleared for redistribution. learning rate learning rate about this quote: RULED, NOT PREVIOUSLY ADJUDICATED. This quote carries an unmapped font glyph and was not covered by the 2026-08-25 ruling, which reached only rows flagged with glyph damage. The same reasoning applies and the same decision follows: THE QUOTE IS NOT EDITED. The unmapped characters are the ones carrying the claim, and no code-to-character table is possible - in one sentence of this corpus the same code is '=' once and 'tau' twice. Repair needs the source PDF and is a re-staging job, costed and refused (0 of 114 appendable; a per-record re-stage fixes 21 and breaks 223). Three sibling rows were found ALREADY ruled under this same pass and are untouched. Quotation withheld because this record is not cleared for redistribution. GAE parameter GAE parameter about this quote: RULED, NOT PREVIOUSLY ADJUDICATED. This quote carries an unmapped font glyph and was not covered by the 2026-08-25 ruling, which reached only rows flagged with glyph damage. The same reasoning applies and the same decision follows: THE QUOTE IS NOT EDITED. The unmapped characters are the ones carrying the claim, and no code-to-character table is possible - in one sentence of this corpus the same code is '=' once and 'tau' twice. Repair needs the source PDF and is a re-staging job, costed and refused (0 of 114 appendable; a per-record re-stage fixes 21 and breaks 223). Three sibling rows were found ALREADY ruled under this same pass and are untouched. Quotation withheld because this record is not cleared for redistribution. clipping parameter clipping parameter about this quote: RULED, NOT PREVIOUSLY ADJUDICATED. This quote carries an unmapped font glyph and was not covered by the 2026-08-25 ruling, which reached only rows flagged with glyph damage. The same reasoning applies and the same decision follows: THE QUOTE IS NOT EDITED. The unmapped characters are the ones carrying the claim, and no code-to-character table is possible - in one sentence of this corpus the same code is '=' once and 'tau' twice. Repair needs the source PDF and is a re-staging job, costed and refused (0 of 114 appendable; a per-record re-stage fixes 21 and breaks 223). Three sibling rows were found ALREADY ruled under this same pass and are untouched. Quotation withheld because this record is not cleared for redistribution. value function coefficient value function coefficient about this quote: RULED, NOT PREVIOUSLY ADJUDICATED. This quote carries an unmapped font glyph and was not covered by the 2026-08-25 ruling, which reached only rows flagged with glyph damage. The same reasoning applies and the same decision follows: THE QUOTE IS NOT EDITED. The unmapped characters are the ones carrying the claim, and no code-to-character table is possible - in one sentence of this corpus the same code is '=' once and 'tau' twice. Repair needs the source PDF and is a re-staging job, costed and refused (0 of 114 appendable; a per-record re-stage fixes 21 and breaks 223). Three sibling rows were found ALREADY ruled under this same pass and are untouched. Quotation withheld because this record is not cleared for redistribution. entropy bonus coefficient entropy bonus coefficient about this quote: RULED, NOT PREVIOUSLY ADJUDICATED. This quote carries an unmapped font glyph and was not covered by the 2026-08-25 ruling, which reached only rows flagged with glyph damage. The same reasoning applies and the same decision follows: THE QUOTE IS NOT EDITED. The unmapped characters are the ones carrying the claim, and no code-to-character table is possible - in one sentence of this corpus the same code is '=' once and 'tau' twice. Repair needs the source PDF and is a re-staging job, costed and refused (0 of 114 appendable; a per-record re-stage fixes 21 and breaks 223). Three sibling rows were found ALREADY ruled under this same pass and are untouched. Quotation withheld because this record is not cleared for redistribution. 14.7% — average relative makespan error of the trained scheduler on the 100 in-distribution test instances Quotation withheld because this record is not cleared for redistribution. over 20% — average relative error of every dispatching rule except MOR on the same test set Quotation withheld because this record is not cleared for redistribution. 12.6% to 20.9% — range of the scheduler's average relative error across the published benchmark families Quotation withheld because this record is not cleared for redistribution. 4.4% and 2.7% — the two cases where the scheduler was second-best by instance size (20x30 and 20x50), and by how much it trailed the best rule Quotation withheld because this record is not cleared for redistribution. R2 = 0.9984 — fit quality of the quadratic model relating runtime to the number of jobs Quotation withheld because this record is not cleared for redistribution. R2 = 0.9815 — fit quality of the linear model relating runtime to the number of machines Quotation withheld because this record is not cleared for redistribution. K = 3 — number of stacked embedding layers (hops of message passing) used in the GNN Quotation withheld because this record is not cleared for redistribution. 100 — number of held-out test instances sampled from the training distribution Quotation withheld because this record is not cleared for redistribution. 20 machines and 100 jobs — size of the largest instance the scheduler was tested on Quotation withheld because this record is not cleared for redistribution. 20 — episodes collected per PPO parameter update Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1080/00207543.2020.1870013.