1. Introduction

2. Materials and Methods

2.2. Experimental Setup

The foundation of our experiment is a simulator designed for formaldehyde production [5]. This simulator has been tailored to emulate the environment of a control room. A significant addition to this setup is the "support panel", which effectively transforms the simulator into a comprehensive control room simulation. This panel encompasses various features: a graphical representation for production monitoring, an alarm list, a procedure list, and an automated suggestion box. Regarding the process, the plant approximates a production rate of 10,000 kg/hr of 30% formaldehyde solution, generated via the partial oxidation of methanol with air. The simulator comprises six sections, namely: Tank, Methanol, Compressor, Heat Recovery, Reactor, and Absorber. With the inclusion of 80 alarms of varying priority levels, the simulator also accounts for nuisance alarms (irrelevant alarms). The simulator’s main screen is visible in Figure 1 and the detail tank mimic can be seen in Figure 3. To test the efficiency of our decision support, we created three scenarios:

• Pressure indicator control failure. In this scenario, the automatic pressure management system in the tank ceases to function. Consequently, the operator must manually modulate the inflow of nitrogen into the tank to preserve the pressure. During this scenario, the cessation of nitrogen flow into the tank results in a pressure drop as the pump continues to channel nitrogen into the plant.
• Nitrogen valve primary source failure. This scenario is an alternative version of the first one. In this case, the primary source of nitrogen in the tank fails. The operator has to switch to a backup system. While the backup system starts slowly the operator has to regulate the pump power to slow down the drop of pressure inside the tank.
• Temperature indicator control failure in the Heat Recovery section. The operator initially attempts to rectify the issue by manually adjusting the set point of the cooling water flow in the absorber. However, this intervention is ineffective. Consequently, the operator has to contact the supervisor for further guidance. The supervisor informs the operator that the fault is beyond the scope of control room resolution and requires the intervention of a field operator. While the field operator is dispatched to address the issue on-site, the control room operator is tasked with managing the reactor’s temperature. The primary concern in this scenario is the potential for the reactor to overheat. To mitigate this risk, the operator must closely monitor and adjust the cooling water temperature of the reactor.

Figure 1. Process flow diagram of the Production. The formaldehyde is synthesized by combining methanol and compressed air, heating the mixture, initiating a chemical reaction in the Reactor, and finally diluting the solution in the Absorber to obtain the appropriate concentration. At the bottom is the different mimic that the operator can open on another screen for a process flow diagram of a specific part of the plant (see Figure 3).

Figure 1. Process flow diagram of the Production. The formaldehyde is synthesized by combining methanol and compressed air, heating the mixture, initiating a chemical reaction in the Reactor, and finally diluting the solution in the Absorber to obtain the appropriate concentration. At the bottom is the different mimic that the operator can open on another screen for a process flow diagram of a specific part of the plant (see Figure 3).

Figure 2. Overview of the Tank section. On the left, we can see the nitrogen flow control panel. In the middle, is the process flow diagram of the tank with all the possible alarms. On the right is the graph of the physical value that needs to be monitored by the operator.

Figure 2. Overview of the Tank section. On the left, we can see the nitrogen flow control panel. In the middle, is the process flow diagram of the tank with all the possible alarms. On the right is the graph of the physical value that needs to be monitored by the operator.

Figure 3. Support Panel Layout: The upper left quadrant displays the alarm list, alerting the operator to current issues. The upper right is the traditional procedure system. The lower left features a critical graph related to production metrics. In the lower right, the suggestion box automatically provide situation-specific recommendations to the operator.

Figure 3. Support Panel Layout: The upper left quadrant displays the alarm list, alerting the operator to current issues. The upper right is the traditional procedure system. The lower left features a critical graph related to production metrics. In the lower right, the suggestion box automatically provide situation-specific recommendations to the operator.

3. Statistical Test

4. Result

4.1. Construction of the Model

In this section, we provide a detailed explanation of the process involved in constructing the dynamic influence diagram. This diagram can be seen in Figure 7 and Figure 8. The primary objective of this model is to detect anomalies and offer the operator an optimal procedure to follow. The simulator is not publicly available but the models and the code to use the model are available at https://github.com/CISC-LIVE-LAB-3/Decision_support

4.1.1. Operational Framework and Objectives of the DID Model

The DID constructed for this research is tailored to address the three scenarios delineated in section 2.1, with a particular focus on the tank system within the process. For the initial two scenarios, the DID provides substantial decision support by simulating the tank system’s physical processes, thereby guiding operators toward maintaining systemic equilibrium. The model’s primary objective is to balance the nitrogen injection rate and the pump power, thereby preventing the risks associated with both overpressure (explosion) and underpressure (implosion). By doing so, the DID facilitates the maintenance of optimal pressure levels inside the tank, which is crucial for the plant’s stability and operational efficiency.

However, for the third scenario, the complexity of modeling the entire process was deemed impractical within the scope of this work. Instead, a simplified model was developed specifically to provide decision support for this scenario. While this streamlined model does not encompass the full intricacies of the process, it is designed to effectively detect anomalies and assist the operator in making the necessary adjustments, particularly concerning the reactor’s temperature. This approach ensures that the operator receives targeted and relevant guidance, enabling them to respond to the scenario with appropriate actions without the need for a comprehensive model of the entire system.

4.1.2. Fault detection

Fault detection in our system is carried out through conflict analysis, as explained earlier. The models for detecting faults in the first two scenarios and the last scenario are illustrated in Figure 9 and Figure 10, respectively. Both models function in a similar manner, but in the first two scenarios, there are additional elements like the "System" node and "Pressure T+1". These elements indicate the option to switch to a backup system in scenarios 1 and 2, which adjusts the nitrogen flow based on the pressure readings.

In both models, the default setting is the auto mode with a predetermined flow. A fault is indicated when the actual flow is anomalously low, leading to a conflict between the expected and actual flow readings. To resolve this, the "Fault" node is adjusted to "Fault control", aligning the low nitrogen flow with the fault condition and thus resolving the conflict. If the issue continues even after reverting to auto mode, a new conflict arises, which is then resolved by adjusting the fault value Fault valve".

4.1.3. Parameter and Structure Specification

The parameter specification relies on these physical equations related to the process as outlined in [5] and the logic of the model. Owing to the discretization of the variables, we utilize a sampling technique to formulate the resultant Conditional Probability Table (CPT). To demonstrate this methodology, we consider the calculation of pressure from our case study.

The pressure is contingent on two variables, Mg (mole) and VG ($m^3$), abiding by the perfect gas law:

$$\mathrm{PV}=\mathrm{nRT}$$

(11)

In our experiment, the corresponding physical equation is:

$$\mathrm{P}=\mathrm{Mg}\ast 8.314\ast 1000\ast 298/(28\ast \mathrm{VG})$$

(12)

where P is expressed in Pascal, MG in Kg, and VG in $m^3$. The temperature, 298, is in degrees Celsius, 8.314 J/(mol*K) is the perfect gas constant, and 28 is the molecular mass of methanol (kg/kmol), which is divided by 1000 to convert to (kg/mol).

We build the structure of the model according to the formula. The pressure depends on Mg and VG. We link them to the Pressure node. We also use this formula to set the expression in the pressure node. The graph can be seen in Figure 20. Additionally, it is worth mentioning that Mg and VG serve as intermediate variables in order to avoid directly connecting all variables to the pressure node, which would result in a too-large conditional probability table. It has also the benefit of displaying the model of the different physical equations in a comprehensible way.

For the CPT generation, we employed a sampling method. In our study, we generate 25 values within each interval for the parent nodes Mg and VG. We then calculated the probability of a value falling within the "Pressure T+1" state intervals after applying the relevant formula. This methodology of sampling and probability estimation is similarly applied to other nodes like Mg, VG, Level of Liquid, Flow of Liquid, Intermediate Node, and Temperature of the Reactor, each governed by their respective physical equations.

The CPT for "Nitrogen Flow" and "Pump Power" depend on the state of their parent nodes. For example, if the system is in "Auto" mode without any faults, the "Nitrogen Flow" is typically around 3.9–4m³/h. This is a normal range for when the system automatically controls the flow to work best. The "Pump Power" also changes based on its parent nodes. These logical CPTs help the Bayesian network to mimic the system’s behavior in different situations, making it a useful tool for decision-making and predicting future scenarios.

Figure 11. Exemple for the calculation of the Pressure CPT.

Figure 11. Exemple for the calculation of the Pressure CPT.

4.1.4. Utility

In our model, we use nodes to represent possible outcomes, such as incidents that could occur in the industrial plant. After inputting observed values and decisions into the influence diagram, we calculate the chances of these outcomes. Each outcome, or consequence node, is linked to a utility node that shows the financial cost of that outcome. For example, the cost of a tank explosion might be around one million dollars, based on sources like [5]. The likelihood of such an explosion depends on factors like pressure and flow rates, and every decision made in the model affects these probabilities.

The model specifically addresses outcomes related to tank pressure, such as "Tank explosion" and "Tank implosion." These are directly linked to pressure changes. For example, the chance of a tank explosion increases linearly as pressure rises. If the pressure is in a safe range, say "101200-102600" Pa, the explosion risk is zero. But if it drops below a critical point, like "-inf-96800" Pa, the explosion risk becomes certain. The "Tank implosion" risk works similarly but increases as pressure drops. This setup helps predict and manage the risks associated with pressure changes in the tank.

4.1.5. Dynamic Model

We use a DID in our model to predict future states of the system and identify actions that maximize utility at each moment. This approach ensures that the chosen actions are optimal, taking into account both current and future scenarios. The model operates across 10 time steps, with each step representing one minute. This allows the model to project the system’s state in one-minute intervals for the next 10 minutes, effectively capturing the dynamics of the system. Such a setup enables us to inform the operator about potential critical events in the next 10 minutes and suggest the best course of action to prevent or manage these events.

In our model, decision-making is closely linked to the states of various nodes, such as "Auto" and "Set Point," which are influenced by the current pressure ("T_Pressure T+1"). This setup mimics how an operator would make decisions based on current and future pressure states, enhancing the model’s realism and predictive capability. The DID is designed to model scenarios where the operator makes correct decisions based on this information. The model also establishes a cause-and-effect relationship between the "Auto" and "Set Point" nodes, where the "Auto" mode automatically sets the "Set Point" to a specific value. This is essential for modeling the system’s response to different operational modes and affects the range of possible actions and their outcomes. Overall, the model provides a detailed representation of the decision-making environment, considering various factors like current conditions, future projections, and the balance between automated and manual controls. This comprehensive approach makes the DID a powerful tool for simulating and enhancing decision-making in complex systems.

4.2. AI Framwork

In the proposed framework, we integrate the predictive prowess of a dynamic influence diagram (DID) with the adaptive capabilities of Reinforcement Learning (RL) agents. The DID is meticulously constructed upon the physical equations governing the process, with a focus on the dynamics of the tank system and other relevant components for specific scenarios. This model is not only designed to capture the physical dynamics of the tank system but also to account for uncertainties inherent in the process. It is adept at identifying anomalies, representing fault states of the tank, forecasting future states, and recommending optimal actions.

A pivotal aspect of employing Bayesian networks, such as the DID, is the discretization of variables, as influence diagrams typically perform more effectively with discrete variables. To address the limitations posed by discretization, we employ localized reinforcement learning agents. These agents refine the actions suggested by the DID, providing continuous values instead of broad intervals. This precision aids the operator in making more accurate decisions.

However, safety concerns necessitate a cautious approach when integrating black-box models like RL agents. Therefore, the continuous value suggested by an RL agent is only utilized if it falls within the DID-recommended interval. If the RL-derived value is outside this interval or appears contradictory, the DID’s recommendation is prioritized by default. This approach ensures that the system’s recommendations remain within safe operational parameters, leveraging the strengths of both the DID and RL agents to enhance decision-making while maintaining a safety-first perspective.

It operates within the Human-in-the-Loop (HITL) setting, employing a Multi Specialized Reinforcement Learning Agent (M-SRLA) configuration. Within this setting, multiple agents operate independently, and only a designated agent becomes active to propose the optimal control strategy to the operator when a process abnormality is identified through the influence diagram, as illustrated in Figure 12, taken from [21]. This framework is called "Human-Centered Artificial Intelligence for Safety-Critical Systems" (HAISC).

Algorithm 1: Influence Diagram-based Recommendation Algorithm.

5. Experiment

5.2. Situation Awareness

Situation awareness (SA) is evaluated through a two-pronged approach. Firstly, participants complete the Situation Awareness Rating Technique (SART) questionnaire following each scenario. This self-assessment tool measures the quality of the participant’s awareness of the situation, capturing their ability to monitor, comprehend, and project the status of various elements in the environment.

Secondly, the SA of participants is assessed in real-time during the scenarios using the Situation Present Assessment Method (SPAM). This method involves intermittently querying participants with three targeted questions at different stages of the scenario. These questions are designed to gauge the participants’ attention and understanding of the current state of the system, their projection of future states, and their perception of situational complexity and dynamics. This dual approach allows for a comprehensive assessment of SA, combining reflective self-reporting with in-the-moment evaluations.

The result of the SART questionnaire can be seen in Figure 14. The statistical test Table 1 shows no statistical significance between the two groups.

Based on the results presented in Table 2, and noting that Group G2 consistently shows lower values than Group G1 Table 2, we interpret the data as follows:

• Monitoring:

  –

  The Shapiro-Wilk test indicates non-normal distribution for G1, but normal for G2.

  –

  Significant differences in monitoring are observed, with G4 showing lower levels as indicated by the t-test and Wilcoxon Rank-Sum Test.

• Planning:

  –

  Both groups exhibit normal distribution according to the Shapiro-Wilk test.

  –

  No significant differences in planning, though G2 tends to have slightly lower levels.

• Intervention:

  –

  Shapiro-Wilk test suggests borderline normal distribution for both groups.

  –

  A significant difference is found, with G2 engaging in less intervention than G3.

• SPAM Index:

  –

  Normal distribution is suggested for both groups by the Shapiro-Wilk test.

  –

  The SPAM Index shows a significant difference, with G2 having a lower index than G3.

In summary, Group G2 consistently exhibits lower levels of monitoring, intervention, and overall SPAM Index compared to Group G1. While planning shows no significant difference, the trend of lower values in G2 is consistent across other dimensions.

Figure 15. Bar plot of the variable of the SPAM questionnaire for the two different groups. G1 without decision support and G2 with.

Figure 15. Bar plot of the variable of the SPAM questionnaire for the two different groups. G1 without decision support and G2 with.

5.5. Physiological Data

In this study, we conducted an extensive quantitative analysis of physiological metrics collected using smartwatch technology. Our focus was on three key parameters: heart rate, temperature, and electrodermal activity (EDA). The aim was to identify significant differences across groups and to uncover underlying patterns and variations within the data. Such insights are invaluable for understanding the physiological responses as captured by the wearable device.

For this analysis, we examined data from Group 1, consisting of 14 participants, and Group 2, with 17 participants. Each participant’s mean values for heart rate, temperature, and EDA were calculated across various scenarios. This methodology enables us to assess and compare the average physiological responses of the groups to these scenarios, shedding light on the distinct patterns observed via the smartwatch.

The Table 6 presents the outcomes of our statistical tests across heart rate, temperature, and EDA. For heart rate, although both groups G1 and G2 pass the normality test, a significant difference in variances is detected. The Welch’s T-test indicates a borderline significant difference (p-value = 0.05), a finding echoed by the Wilcoxon test (p-value = 0.09). The temperature measurements didn’t show any significant difference between the two groups. For EDA, neither group shows a normal distribution. The variance is not significantly different, and the Wilcoxon test suggests no significance in their differences.

A notable finding was observed in the comparative analysis of groups G1 and G2. It was discerned that Group 2 tended to have a lower heart rate than Group 1. This outcome is particularly significant as it suggests the potential influence of a recommendation system in alleviating stress among control room operators. The implication is that decision support systems may play a crucial role in reducing stress levels, thereby potentially improving both the well-being and operational efficiency of operators in high-stress environments. Such findings underscore the importance of integrating supportive technologies in workplaces where decision-making under pressure is a regular occurrence.

Figure 18. Box plot of the Heart Rate Across Three Scenarios for Each Group.

Figure 18. Box plot of the Heart Rate Across Three Scenarios for Each Group.

Figure 19. Box plot of the Temperature Across Three Scenarios for Each Group.

Figure 19. Box plot of the Temperature Across Three Scenarios for Each Group.

Figure 20. Box plot of the Electrodermal Activity Across Three Scenarios for Each Group.

Figure 20. Box plot of the Electrodermal Activity Across Three Scenarios for Each Group.

6. Discussion

7. Conclusions

Abbreviations

References

1. Proudlove, N.C.; Vaderá, S.; Kobbacy, K.A.H. Intelligent management systems in operations: a review. Journal of the Operational Research Society 1998, 49(7), 682–699. [CrossRef]
2. Kobbacy, K.A.H.; Vadera, S. A survey of AI in operations management from 2005 to 2009. Journal of Manufacturing Technology Management 2011, 22(6), 706–733. [CrossRef]
3. Eom, S.; Kim, E. A survey of decision support system applications (1995–2001). Journal of the Operational Research Society 2006, 57, 1264–1278. [CrossRef]
4. Power, D.J. Decision support systems concept. In Business Information Systems: Concepts, Methodologies, Tools and Applications; IGI Global: 2010; pp. 1–5. [CrossRef]
5. Demichela, M.; Baldissone, G.; Camuncoli, G. Risk-based decision making for the management of change in process plants: benefits of integrating probabilistic and phenomenological analysis. Industrial & Engineering Chemistry Research 2017, 56(50), 14873–14887. [CrossRef]
6. McNaught, K.; Chan, A. Bayesian networks in manufacturing. Journal of Manufacturing Technology Management 2011, 22(6), 734–747. [CrossRef]
7. Hsieh, M.-H.; Hwang, S.-L.; Liu, K.-H.; Liang, S.-F. M.; Chuang, C.-F. A decision support system for identifying abnormal operating procedures in a nuclear power plant. Nuclear Engineering and Design 2012, 249, 413–418. [CrossRef]
8. Lee, S.J.; Seong, P.H. Development of an integrated decision support system to aid cognitive activities of operators. Nuclear Engineering and Technology 2007, 39(6), 703. [CrossRef]
9. Valdez, A.C.; Brauner, P.; Ziefle, M.; Kuhlen, T.W.; Sedlmair, M. Human factors in information visualization and decision support systems. 2016, Gesellschaft für Informatik e.V. [CrossRef]
10. Madhavan, P.; Wiegmann, D.A. Effects of information source, pedigree, and reliability on operator interaction with decision support systems. Human factors 2007, 49(5), 773–785. [CrossRef]
11. Al-Dabbagh, A.W.; Hu, W.; Lai, S.; Chen, T.; Shah, S.L. Toward the advancement of decision support tools for industrial facilities: Addressing operation metrics, visualization plots, and alarm floods. IEEE Transactions on Automation Science and Engineering 2018, 15(4), 1883–1896. [CrossRef]
12. Naef, M.; Chadha, K.; Lefsrud, L. Decision support for process operators: Task loading in the days of big data. Journal of Loss Prevention in the Process Industries 2022, 75, 104713. [CrossRef]
13. Kjærulff, U.B.; Madsen, A.L. Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis. Springer: 2013.
14. MIETKIEWICZ, Joseph, et al. Dynamic Influence Diagram-Based Deep Reinforcement Learning Framework and Application for Decision Support for Operators in Control Rooms. 2023. [CrossRef]
15. MIETKIEWICZ, Joseph; MADSEN, Anders Læsø. Data driven Bayesian network to predict critical alarm. In: European Conference on Safety and Reliability. Research Publishing, Singapore, 2022. p. 522. [CrossRef]
16. HUGIN EXPERT A/S, Hugin sofware https://www.hugin.com.
17. Abbas, A. N., Chasparis, G. C., & Kelleher, J. D. (2023). Hierarchical framework for interpretable deep reinforcement learning-based predictive maintenance. Data & Knowledge Engineering, 102240. [CrossRef]
18. Abbas, A. N., Chasparis, G. C., & Kelleher, J. D. (2022, July). Interpretable Input-Output Hidden Markov Model-Based Deep Reinforcement Learning for the Predictive Maintenance of Turbofan Engines. In International Conference on Big Data Analytics and Knowledge Discovery (pp. 133-148). Cham: Springer International Publishing. [CrossRef]
19. Abbas, A. N., Chasparis, G. C., & Kelleher, J. D. (2023). Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces. arXiv preprint arXiv:2310.14788. [CrossRef]
20. Abbas, A. N., Chasparis, G. C., & Kelleher, J. (2022). Deep Residual Policy Reinforcement Learning as a Corrective Term in Process Control for Alarm Reduction: A Preliminary Report. [CrossRef]
21. Mietkiewicz, J., Abbas, A. N., Amazu, C. W., Madsen, A. L., & Baldissone, G. (2023). Dynamic Influence Diagram-Based Deep Reinforcement Learning Framework and Application for Decision Support for Operators in Control Rooms. [CrossRef]
22. Spielberg, S., Tulsyan, A., Lawrence, N. P., Loewen, P. D., & Gopaluni, R. B. (2020). Deep reinforcement learning for process control: A primer for beginners. arXiv preprint arXiv:2004.05490. [CrossRef]
23. François-Lavet, V., Henderson, P., Islam, R., Bellemare, M. G., & Pineau, J. (2018). An introduction to deep reinforcement learning. Foundations and Trends® in Machine Learning, 11(3-4), 219-354. [CrossRef]
24. Fujimoto, S., Hoof, H., & Meger, D. (2018, July). Addressing function approximation error in actor-critic methods. In International conference on machine learning (pp. 1587-1596). PMLR.
25. Shapiro, S.S.; Wilk, M.B. An analysis of variance test for normality (complete samples). Biometrika 1965, 52(3/4), 591–611. [CrossRef]
26. Mann, H.B.; Whitney, D.R. On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics 1947, 50-60. [CrossRef]
27. Levene, H. Robust tests for equality of variances. In Contributions to Probability and Statistics; 1960, 278-292.
28. Welch, B.L. The generalization of ‘STUDENT’S’ problem when several different population variances are involved. Biometrika 1947, 34(1-2), 28-35. [CrossRef]
29. Student. The probable error of a mean. Biometrika 1908, 6(1), 1-25. [CrossRef]
30. Mietkiewicz, J.; Madsen, A.L. Enhancing Control Room Operator Decision Making: An Application of Dynamic Influence Diagrams in Formaldehyde Manufacturing. In European Conference on Symbolic and Quantitative Approaches with Uncertainty; 2023; Springer Nature Switzerland: Cham, pp. 15-26. [CrossRef]
31. Weidl, G.; Madsen, A.L.; Israelson, S. Applications of object-oriented Bayesian networks for condition monitoring, root cause analysis and decision support on operation of complex continuous processes. Computers & Chemical Engineering 2005, 29(9), 1996–2009. [CrossRef]
32. Dey, S.; Story, J.A.; Stori, J.A. A Bayesian network approach to root cause diagnosis of process variations. International Journal of Machine Tools and Manufacture 2005, 45(1), 75-91. [CrossRef]
33. Energy Institute, London. Human Factors Performance Indicators for the Energy and Related Process Industries. 1st edition; Energy Institute: London, 2010; ISBN 978-0-85293-587-3.
34. Amazu, C.W.; et al. Analysing "Human-in-the-Loop" for Advances in Process Safety: A Design of Experiment in a Simulated Process Control Room. 2023. [CrossRef]