Artificial Intelligence to Solve Production Scheduling Problems in Real Industrial Settings: Systematic Literature Review

1. Introduction

The adoption of Industry 4.0 principles, the advancement of smarter factories, and the integration of intelligent sensors and interconnectivity across various organizational components have contributed to the expanding volume of literature regarding the application of Artificial Intelligence (AI) in manufacturing systems. This field of study has seen rapid growth in recent years. Contemporary manufacturing settings are affected by numerous factors that impact the production process [1], including machine failures [2,3], order fluctuations, and unpredictable job arrivals. To be competitive in the actual context is important to be flexible and be able to respond faster to variations in production planning [4]. Currently, production procedures are dynamically changed to actively satisfy consumer wants and create a wide range of products. To this purpose, the manufacturing ecosystem of today is distinguished by a reduced product life cycle, a high level of product variability, and an escalating level of international competition [5]. AI is an important instrument in the context of manufacturing systems to respond fast and predict future anomalies in the production plan; the AI instrument can be used as support for the decision-making process. In literature, there are a lot of contributions about the use of AI instruments to realize dynamic scheduling [6,7] algorithms or algorithms able to find difficult correlations between factors in the manufacturing environment.

The development of a dynamic scheduling programme based on AI is the major objective of the European project AIDEAS's "Fabrication Optimiser" tool, which was born in this context. Consequently, examining how other authors had addressed similar challenges was essential.

This paper has a twofold objective:

-

Understand what the trends are in solving scheduling problems through the use of AI and what AI techniques are most widely used in the literature

-

Analyse how other authors solve production scheduling problems in real industrial settings and see what advantages they have achieved for the companies where the solutions have been implemented.

Thus, a systematic literature review was conducted using the Scopus database and bibliometric tools such as VOSviewer [8]

The scheduling problem is a classic NP-hard problem [9] and is also one of the key links for the efficient operation of an intelligent production system because dynamic scheduling can optimize several KPIs in production space, for example, reduce the tardiness [10], the cost of storage [11], makespan [12], travelling time [13] and others KPI that change from company to company. Intelligent production gave numerous advantages in terms of flexibility, maintainability, and cost. AI is not used only for dynamic scheduling but is used in production plans to help in the decision-making process. However, it is important to emphasise that production scheduling problems are classified into several subsets. The main scheduling problems are:

-

Single Machine Scheduling Problem (SMSP) [14]. SMSP regard the allocation of a set of tasks in a single machine in order to optimize an objective function.

-

Flow-Shop Scheduling Problem (FSSP) [15]. In a FSSP there are a set of tasks that must be scheduled in a set of machines. In this type of problem, the items to be produced must follow a precise sequence of tasks, so each task will have a precedence constraint with other tasks. All the items to be scheduled must follow the same manufacturing sequence so the flow of material and information in this type of problem is unidirectional.

-

Job-Shop Scheduling Problem (JSSP) [16]. A JSSP is similar to the FSSP, there will be a set of items that will have to be processed on a set of machines. However, unlike the FSSP here the items do not necessarily have the same manufacturing sequence, so the flow of materials will be multi-directional.

-

Open-Shop Scheduling Problem (OSSP) [17]. Also, in the OSSP, there will be a set of elements that must be processed on a set of machines, but in this case, there are no precedence constraints between the activities to be performed.

-

Parallel Machine Scheduling Problem (PMSP) [18]: PMSP involves scheduling a set of jobs to be processed on multiple machines simultaneously or in parallel. The primary objective is to determine how to allocate jobs to machines and in what order. If all machines have the same processing speed and capabilities it is called Identical PMSP, if the machines are grouped into classes, and machines within the same class have the same processing speed are called Uniform PMSP. Meanwhile, if each machine has a unique processing speed is called Unrelated PMSP.

These are, in short, the main scheduling problems; we will speak of Flexible JSSP (FSSP) [19]or Flexible FSSP (FFSSP) [20] when the scheduling problem combines one of the aforementioned problems with PMSP.

The dynamic nature of the manufacturing systems implies the necessary adoption of a dynamic scheduling paradigm to deal with unforeseen events that disrupt the execution of a schedule as the assigned apparitions can be immediately redirected to other machines. According to Elbasheer et al. [21], there are three major manifestations of dynamic scheduling in AI literature: task re-scheduling concerns the reprogramming of a specific activity within the production process as a reaction to an interruption in the original program. Resource allocation especially in flexible shop floors where the use of AI should improve the ability to allocate resources to deal with plan disruptions and line balancing after any interruptions in the production process.

In this paper, Section 2 explains how the research was conducted and which tools were used to study publication trends, and which AI techniques are most prevalent in the literature. In Section 3, relevant contributions found in the literature are reported. Section 4 provides discussions, Section 5 reports future development, and a conclusion.

2. Methods and Data

The literature review was conducted in order to respond to the two main scopes of this study; analyse what are the trends in the use of AI to solve scheduling problems and understand what techniques are the most widely used and study how authors have solved industrial settings and what benefits they have brought to the companies.

Figure 1 shows the steps taken to carry out the literature review. In the following sections, all these steps are explained in detail.

2.1. Preliminary Research

The literature search is set using Scopus as a database and searching for articles, conference papers and reviews, reviews, and book chapters in English published.

A preliminary search was conducted searching on Scopus for the keywords “Scheduling” AND “Artificial Intelligence” in titles, abstracts, and keywords. Scheduling problems managed using AI is an attractive topic for scientific debate in various fields with an increase in the number of publications, as shown in Figure 2.

The search for this analysis focused on how authors solve production scheduling problems using AI techniques, limiting the analysis to articles published from 2011 (the year of the advent of I4.0) to 2022 and 4859 papers were found. A filter was used on the publication of the engineering domain only to exclude some out-of-scope publications; this resulted in 1734 papers written in English. Then, a bibliometric analysis was conducted on these articles.

2.2. Bibliometric Analysis

One of the research questions was to identify which AI techniques are most commonly used to solve scheduling problems and for what purpose. To conduct a specific literature review, it is important to know the specific keywords to search for in order to quickly identify useful papers in the Scopus database. For this aim, it was used VOSviewer, a bibliometric software that can find the most frequent occurrences present in a big database of publications.

Thus, it has been possible to find common correlations among several keywords. In particular, only keywords capable of leading to AI techniques were selected. A minimum threshold of keyword occurrences of 10 has been set to exclude less frequent keywords.

In Figure 3 is possible to see the main AI techniques used to solve scheduling problems like, for example, Neural Networks (NN), Deep Neural Networks (DNN), Reinforcement Learning (RL), Swarm Intelligence (SI), Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), Decision Trees (DT).

2.3. Specific search

From the results of bibliometric analysis, a more specific search on Scopus was conducted searching scheduling and AI techniques as keywords in order to see the number of publications.

Figure 4 shows the number of publications for each couple of keywords searched to see what AI techniques the most are widespread. This research shows that the largest contributions in the literature concern the use of PSO, NN and RL for solving scheduling problems.

The next stage of the research is to analyze how the authors solve production scheduling problems in real industrial settings through the use of PSO, NN and/or RL algorithms. For this reason, a specific search on the Scopus database was conducted searching in title, abstract and keywords: “Scheduling” AND (“Particle Swarm Optimization” OR “Neural Network” OR “Reinforcement learning”) AND “case study”.

This choice was made because this analysis aims to analyze which benefits and advantages companies have obtained from such algorithms, thus excluding all articles that illustrate an algorithm without application cases.

Only articles and conference papers written in English and published from 2011 to 2022 in the engineering domain were considered. A total of 366 publications were found reading titles and abstracts. Publications that do not concern the scheduling of production orders within an industrial context (e.g., energy storage and distribution, urban transport planning etc.) were excluded. Publications that illustrate the algorithm and test it on a simulation plant without reporting the benefits obtained from the application of AI techniques are also excluded [22]. A total of 22 papers were found that applied NN, PSO or RL to solve scheduling problems in industrial case studies.

3. Literature review of relevant papers

This section reports on contributions by other authors to solve the scheduling problem within production sites using PSO, NN and/or RL. This research focuses on understanding how the authors used these techniques and what benefits they brought to the companies.

3.1. Particle Swarm Optimization

PSO is an optimisation technique inspired by the social behaviour of birds and fish. In the context of production scheduling problems, PSO algorithms mimic the collective intelligence and cooperation observed in these natural systems to find optimal solutions. PSO offers a dynamic approach to tackle complex scheduling challenges, with the aim of improving efficiency and minimising production costs.

In PSO, a population of potential solutions, represented as particles, iteratively explores the solution space. Each particle adjusts its position and velocity based on its own experience and that of its neighbours. This cooperative search mechanism allows the PSO to efficiently navigate the vast solution space of scheduling problems.

Researchers and practitioners have successfully applied PSO to a variety of scheduling problems, including job sequencing, resource allocation, and execution time minimisation. By taking advantage of PSO's adaptability and flexibility, manufacturing companies can achieve greater scheduling accuracy and better operational performance, reducing costs and increasing productivity. PSO's ability to handle both single-objective and multi-objective scheduling problems makes it a valuable tool for the manufacturing industry.

To solve a particular scheduling problem, Wang et al. [23] develop a two-stage optimization method to improve the energy efficiency of a FJSSP. The first phase involves the use of a GA to optimize the selection of machine tools for the production process. The second phase combines PSO with GA to improve the sequence of operations. In this combined approach, the GA helps to improve the global exploration capability to avoid early convergence problems in the PSO. The proposed algorithm was evaluated in a practical case study, achieving an 8.5% reduction in production costs and 10.2% reduction in energy consumption compared to the scheduling programs previously employed by the company tested. A different hybrid approach was developed by Chen et al. [24] who realize an algorithm combining variable neighborhood search and PSO to solve PMSP in the solar cell industry. In the proposed case study, variable neighborhood search is used to decide in which order tasks are to be performed and PSO is used to decide the assignment of machines for all production orders. The proposed solution is better than the traditional PSO and the heuristic algorithm used by the company under investigation and achieves the solution of the scheduling problem in 43.16 seconds, faster than the other two solutions. Du et al. [25] propose a combination of PSO with artificial immune to solve an assembly JSSP to minimize the completion time. The algorithm was tested in a real case study and proved to find an optimal solution to the problem in only 106 seconds, faster than the previous approach used by the company. Hecker et al. [26] designed two algorithms to solve the scheduling problem in the bakery industry: the first one uses PSO meanwhile the second uses ACO algorithm. A comparison of the two algorithms was conducted and shows that PSO is faster (39s when optimising makespan and 15s when optimising total machine idle time in the average calculation time) than ACO and returns better results in both optimisation problems. A re-entrant two-stage FSSP where all jobs must visit two times the sequence of the production process was solved by Huang et al. [27] using farness PSO (FPSO). FPSO differs from traditional PSO in that swarm behaviour learns from experience and improves the solution from the self-owned and distant population. The method was tested in a real case and the results were compared with results from traditional PSO and ACO. FPSO outperforms both approaches providing an average improvement in effectiveness of 39.47% and 42.99% compared to PSO and ACO for small-scale problems. Ramezanian et al. [28] designed a PSO algorithm to solve lot size and scheduling problems in a tile industry. The problem is a classic four-stage FFSSP, and the objective function was to find the minimum cost of production, inventory, and external acquisition. The proposed algorithm gave a scheduling program and lot-size in 479 s, an acceptable time for the company to solve a large optimization problem. A production schedule and maintenance which considers energy cost, machine production efficiency, and production target were developed by Sun et al. [29]. The PSO model presented was tested in one company and involves the implementation of joint energy and maintenance management. The implementation generated a reduction in production costs compared to the previously used approach. A different use of PSO is given by Mohammadi et al. [30] who propose a combination of PSO and ε-constraint method, a multi-objective decision-making method. It is considered a 'make-to-order' production system, responsible for the production and transportation of customer orders, the described problem is a combination of a FJSSP and a vehicle routing problem. The proposed scheduling algorithm is a bi-objective mixed integer model that can find a solution that minimizes production and transport costs and the weighted sum of delivery earliness and tardiness. A bi-objective algorithm to solve FJSP with uncertain processing times was developed by Li et al. [31] too. They realize a combination of GA and binary PSO in order to minimize the makespan and a value of deviation from the expected makespan. The proposed method was tested in 9 case studies and performed better in terms of robustness than the stochastic method and a conventional method such as the hybrid GA. Wang J et al. [32] realize a bi-objective scheduling optimization method for a single machine (SMSP) that minimizes energy consumption and total tardiness through the use of PSO. The algorithm was tested on a CNC machine and returns multiple solutions with different values of energy consumption or tardiness that support the process planner his choice.

3.2. Neural Networks

NN used in production scheduling exploits artificial neural networks, a subset of machine learning, to improve the planning and optimization of production processes. These networks are trained using historical data and are designed to predict and optimize various elements of scheduling, such as resource allocation, job sequencing and production timing. NN demonstrate their adaptability in dealing with complex and dynamic production environments, increasing the accuracy and efficiency of scheduling.

An artificial NN algorithm to track the energy consumption of CNC machines was developed by Wang et al. [23]. The proposed algorithm is combined with a multi-objective optimisation model for the production re-scheduling process that minimises energy consumption, makespan and balanced machine utilisation levels. The proposed algorithm was validated on several industrial trials and achieved a 30% improvement in energy consumption and a 50% improvement in productivity. Another interesting approach to the scheduling problem was realized by Zhou et al. [33]. In this case, there is a smart factory with 4 equal workstations, every single station has its schedule program that runs on a distributed computer and realizes the scheduling with a metaheuristic method and has its own NN algorithm that learns from the workstations. The learned knowledge was shared with a centralized computer system where there is a scheduling system based on multi-agent RL (MARL) logic (DQN method) that learns from the 4 workstations and shares this knowledge with the 4 workstations. The proposed solution reduces 11.9% the lead time compared with only DQN algorithm.

Azab et al. [34] develop a framework that combines commercial software tools for scheduling with a machine learning approach to predict machine failure in scheduling programs. The proposed approach was tested in a pharmaceutical company and different AI techniques were tested; the results show that the best performance was given by the use of the Decision Forest algorithm, but the NN algorithm gave better results in predicting the machine failure time.

A model that uses artificial NN to schedule the workforce of a company was designed by Simeunović et al. [35]. The proposed algorithm aimed to predict the number of employees for the following days based on various factors such as customer requests number of working hours, etc. Thanks to this contribution, the waiting time of the company's employees was reduced to 2.3 minutes, leading to an increase in the company's productivity and a higher degree of customer satisfaction.

Wang et al. [36] use ANN to optimize the milling process parameters (energy consumption and surface roughness) for producing one single part. ANN are employed to model intricate non-linear connections between essential process variables and the recorded data of both energy usage and surface quality. Based on the optimized parameters, several intelligent methods, such as Pattern Search, GA and Simulated Annealing are applied to find an optimal sequencing, setting-up and scheduling for multiple machines. In the case study, the Simulated Annealing algorithm was used in two forms. The first model aims to optimise energy consumption and makespan, while the second only optimises energy consumption. With the second approach, there is a reduction in energy consumption of 2795 kJ but an increase in makespan of about 23 min compared to the first.

3.3. Reinforcement learning

In the field of production scheduling, RL involves the use of AI algorithms to make the best possible choices in production operations. RL agents acquire knowledge through practical experience (try and error approach) and engagement in the production environment, with the aim of optimising efficiency, reducing expenses and improving scheduling results. This methodology offers versatility and adaptability in dealing with intricate and ever-changing scheduling dilemmas encountered in manufacturing. RL techniques are use in the production planning and control but mainly to solve production scheduling problems [37], [38].

Wang X, et al. [39] propose a multi-agent RL (MARL) approach to solve a FFSSP with the aim of minimising the makespan value. The problem involves assigning workloads to 18 robot stations in parallel with different processing times. Qmix algorithm was used to learn in the environment and the proposed algorithm outperformed in terms of computational time other classic heuristic approaches and also a Distributed Agent Scheduling Architecture (DASA); another RL approach which differs from MARL because in the latter approach, the reward function is shared among all agents while in the DASA each agent aims to maximise its own reward function. The effectiveness of MARL is also confirmed by the study in the paper [40] which used several single-agent RL (SARL) algorithms for solving the scheduling problem in a human-robot context, a case of SMSP. Again, MARL (here, however, it uses DQN as an algorithm) outperforms other RL algorithms in terms of calculation time, training speed and goodness of solution.

Vijayan S, et al. [41] here tested an RL method exploiting a Q-learning algorithm to solve FSSP, the first is the case of a plastic toy factory, where the algorithm is compared with other metaheuristic approaches and the results are better, in particular, there is a decrease in computation time of up to 18%, outperforms even the PSO. The second is stator core manufacturing; here too there were low computation times and improvements in makespan.

Elsayed, E.K, et al. [42] adopt the Actor-Critic (AC) network’s training algorithm-based RL for achieving the optimal policy of the JSSP. The algorithm was tested in a real case where scheduling was previously done following FIFO logic, the proposed algorithm achieved better results in terms of makespan by going from 97UT to 60UT.

Ghaleb M, et al. [43] proposes an RL-based approach for solving the scheduling problem on three parallel machines (PMSP). These machines are subject to planned and unplanned outages that have a major impact on the scheduling plan. The solution proposed by the authors is an example of a multi-objective scheduling problem, the company wants to maximise the throughput, minimise the mean cycle time, and minimise the number of tardy orders. A Q-learning-based agent is created with the purpose of performing ongoing rescheduling. The agent follows a set of rules, which include obtaining the current status of the production unit, computing the reward for the previous action, choosing the subsequent action, transmitting the newly chosen action to the shop floor, and revising the state-action table with the recently acquired system status. The solution of the proposed approach outperforms the previous EDD rule scheduling method used by the company in terms of total weighted tardiness, throughput and mean cycle time.

Drakaki M, et al. [44] present a combination of Timed Colored Petri Nets (CTPNs) and RL to solve the scheduling problem in a manufacturing plant. The authors propose a CTPNs model to solve the scheduling problem and a Q-learning RL algorithm is used as a guide to improve the solution and reduce the computational time for large-scale problems. The method was tested in a case study to solve a warehouse order-picking scheduling and also applied to known JSSP benchmark examples and compared with other approaches in order to validate the solution.

Said N, et al. [45] introduce an algorithm that utilizes Q-learning optimization for addressing a flexible and dynamic JSSP in a real-world scenario involving a pharmaceutical factory equipped with 18 machines and 22 different products. The algorithm suggested in the study demonstrates its ability to attain efficient scheduling within a brief production cycle, requiring minimal time and without relying on prior scheduling knowledge. This leads to an enhancement in the overall productivity of the factory. The proposed approach reduces the makespan value by 20%-40% (depending on the size of the problem) with respect to a FIFO strategy.

4. Discussion

This section will discuss the results of this research. Thanks to the Scopus database, it was possible to identify PSO, NN and RL as the main approaches. Table 1 is a compilation of the 22 contributions analysed in the previous section.

The first column reports the techniques and algorithms used by the authors and is important to highlight the high adaptability of the approaches with other optimization algorithms or AI techniques. This aspect allows the creation of hybrid algorithms that increase the overall performance of the solution. PSO, NN and RL have been used to solve a very wide range of scheduling problems. In fact, the problems analysed differed greatly from each other (JSSP, SMSP, FSSP, FJSSP, FFSSP, PMSP), which highlights the very high flexibility of the 3 approaches examined. As far as business benefits are concerned, here too they differ. As with any AI or data-driven solution, the results will depend very much on the quality and availability of data on the part of companies. Nevertheless, in the papers analysed, there is a high diversity of benefits, such as reduced makespan, which consequently leads to more efficient production in terms of parts produced per unit of time, or even reduced delays, which are often critical for companies. An important aspect that emerged from the analysis concerns the small number of publications, compared to the large number of papers found, that explicitly report on company benefits. Certainly, this aspect is strongly influenced by the difficulty in accessing and sharing company data, but it is also linked to the fact that some results obtainable from the application of a new scheduling plan cannot be seen in the short term. One aspect that can be analysed immediately is certainly the reduction in calculation time, which in fact is a parameter reported in many cases analysed. Reducing calculation time is also an important aspect in an industrial context because it allows for greater flexibility and the possibility of being able to schedule several times a day and thus be able to respond to any abnormal events. Having the possibility of modifying the scheduling plan in real or near real time is a difficult challenge often in complex contexts with different production constraints. The more complex the problem, the more critical the calculation time will be; in fact, from the RL articles analysed, it was seen that MARL solutions provide better execution times and flexibility than single-agent RL solutions.

5. Conclusions

The following paper was written with the idea of clarifying two objectives:

-

Understand what are the trends in solving scheduling problems through the use of AI and what AI techniques are most widely used in the literature

-

Analyse how other authors solve production scheduling problems in real cases and see what advantages they have achieved.

Thanks to bibliometric analysis and the Scopus database, it was possible to answer the first question and see that the trend of using AI to solve scheduling problems in engineering is growing year after year, with the use of PSO, NN and/or RL being the most widely used approaches in the literature. From this point, a more specific literature review was conducted to see how the authors solve production scheduling problems in real industrial settings through the use of PSO, NN and RL. The three AI techniques present different contributions in which the algorithms are used to solve different types of scheduling problems, classic NP-hard problems, like single-objective or multi-objective in several scenarios like job-shop, flow-shop, and not only. This study showed how, through the use of AI, the companies concerned obtained benefits that can be of different types depending on internal problems.

Future steps will concern the realisation of an algorithm for the optimisation of production scheduling for the pilots of the European AIDEAS project in order to enrich the contribution to the literature