2.1. Experimental Set-Up
Experimental data is acquired by a self developed set-up depicted in Figure 2 of experimental set up. The reaction is carried out in a gradient-less Berty-type reactor built by the FAU Erlangen Nuremberg according to [
11], containing a fixed bed of 4.3g commercial CuO/ZnO/Al
2O
3 catalyst (CZA) from BASF. The catalyst had been ground to 0.315-0.5mm particles to avoid intraparticular transport limitations. The used catalyst was acquired in a former PhD project ([
8]) and was stored at ambient conditions since then. The remaining gas volume of the reactor is approximately 280mL including inert filling material as spherical
-Al
2O
3 (charge no. 5) or 286mL without additional filling (charge no. 6). To verify the assumption of an ideal CSTR behaviour, the rotation speed of the stirrer had been varied during preliminary tests. Above 3000rpm no changes were observed. Thus, the value was set to 4000rpm with additional safety. The stirrer is driven by an electric engine, equipped with permanent magnets that disintegrate when getting in contact with hydrogen or hydrocarbons. Due to constructional reasons, the engine is connected to the reaction chamber. Furthermore, permanent magnets lose their field strength when heated, so the stirrer performance would be permanently negatively affected. For these reasons, it is particular important to cool and flush the engine of the stirrer with nitrogen. This constrains the concentration of the educts, but since nitrogen is an inert for the reaction, it can be used as an indicator for the occurring volume contraction. The experimental set-up is designed for reaction conditions up to 260°C and 60bar.
The utilized gases are supplied from gas bottles with a high purity of 99.999vol.-% for N2, H2 and CO2 and 99.97vol.-% for CO. Bronkhorst mass flow controllers (MFC) are used to dose the educt gases (further information in E.2). Additional piping length is given to mix the gases at the inlet in order to precondition for the reaction. The entire piping periphery around the reactor is heated to avoid condensation.
The experimental set-up is designed to perform steady state and dynamic experiments under isothermal and isobaric conditions. Changes in temperature and pressure during operation are recorded and can be evaluated posterior. However, due to the sheer caloric mass of the reactor, intentional alterations in temperature within an experiment are unfavourable. They greatly increase the required time for an experimental investigation. Therefore experiments are mostly designed to stay at one temperature.
Several temperature probes are located in the reactor. One sensor is situated above and one beneath the catalyst bed. Two further sensors are measuring the heating jacket. Hence, only a small amount of enthalpy is released from the reaction and the great thermal mass of the reactor itself is acting as a dampener, one of the jacket sensors is used to control the reactor temperature. This has proven to be the most stable operation to keep a constant reaction temperature.
A dome pressure regulator (Equilibar) was used to keep a constant pressure in the reactor allowing a discharge during continuous experiments. Hence analytic devices are usually particularly sensitive regarding their inlet pressure, the outlet stream after the dome pressure regulator is first depressurized via a relief valve to 4bar and afterwards further reduced to 1bar using a Swagelok downstream pressure regulator. This method ensures constant pressure at the analytics while inlet conditions to the reactor can be dynamically modified. The total flowrate at the reactor outlet is not measured. If the reactor pressure is increased during an experiment the dome pressure regulator reduces the outlet flow until the pressure level is reached. In this case the analytic flow could be inappropriate.
One part of the automation is managed by Siemens’ PCS7. This ensures the acquisition of plant related data and the operation of actuators. The other part of automation is realized via MATLAB in the interest of applying complex mathematical inlet perturbations, precise timing of new conditions, creating a flexible, user friendly interface and recording data during the time of an experiment for simpler analysis. PCS7 and MATLAB where connected through OPC DA, a proven industrial communication standard.
This allows imposing classical step changes between different constant input values with varying time intervals, as well as periodical perturbation of inlet conditions (e.g. simultaneous modulation of inlet concentration and total flow rate as phase shifted sinusoidal wave form) as shown in Figure 3, making this plant an unique experimental set up. Besides that, another degree of freedom is the use of different fillings of the reactor with fresh catalyst. We refer to this by a catalyst charge number.
Outlet concentrations are quantified in terms of molar fractions of the components CH3OH, CO2, CO, H2, H2O and N2 using a µ-GC from Agilent (model 490). Samples are measured by a method that takes approximately 90s, which had been optimized to execute with precise component determination and fast sampling. This allows an almost continuous collection of data points. Further details about the analytics are summarized in the E.1.
Figure 2.
Scheme of experimental set-up: gas supply from gas bottles via mass flow controllers (MFC’s), Berty type reactor filled with CuO/ZnO/Al2O3 catalyst and analytics (µ-gas chromatography from Agilent). The reactor is monitored and controlled for temperature and pressure. The latter is regulated via a membrane back pressure regulator. Further on the downstream side the pressure is released through a relief valve to 4bar, ensuring a continuous effluent flow. In order to protect the electric engine of the stirrer from the reaction media and heat, N2 is flushed through from the top into the reaction chamber. N2 is used also as an indicator component to quantify the volume contraction caused by the reactions.
Figure 2.
Scheme of experimental set-up: gas supply from gas bottles via mass flow controllers (MFC’s), Berty type reactor filled with CuO/ZnO/Al2O3 catalyst and analytics (µ-gas chromatography from Agilent). The reactor is monitored and controlled for temperature and pressure. The latter is regulated via a membrane back pressure regulator. Further on the downstream side the pressure is released through a relief valve to 4bar, ensuring a continuous effluent flow. In order to protect the electric engine of the stirrer from the reaction media and heat, N2 is flushed through from the top into the reaction chamber. N2 is used also as an indicator component to quantify the volume contraction caused by the reactions.
Figure 3.
Example of a single experiment (MeOH_028). Top Measured output of the reactor as mole fractions. Middle Incoming feed of the reactor as mole fractions and volume flow. Bottom Temperature and pressure of the reactor.
Figure 3.
Example of a single experiment (MeOH_028). Top Measured output of the reactor as mole fractions. Middle Incoming feed of the reactor as mole fractions and volume flow. Bottom Temperature and pressure of the reactor.
2.2. Experimental Procedure
When fresh catalyst was filled into the reactor, it was reduced using 2vol.-% of H
2 in N
2 with a total volumetric flowrate of 500NmL/min at ambient pressure starting from room temperature heating up to 180°C within 2h and keeping this temperature for another 12h according to [
8]. This procedure was repeated after every shutdown of the plant and sometimes additionally to check for the effect on the catalytic performance. Otherwise the catalyst was flushed with nitrogen between methanol synthesis experiments.
To start a methanol synthesis experiment with already preconditioned catalyst the flow of the inert flushing of the stirrer is increased depending on the aimed pressure and the stirrer speed is set to 4000rpm. Afterwards, the inertized reactor is heated up at ambient pressure to the desired temperature. Meanwhile the pressure of the gas supply is adjusted. While N2, H2 and CO are fed directly from gas bottles, CO2 is pressurized with a compressor from Maximator to reach up to 65bar. The CO2 piping is heated to avoid condensation. After the temperature is reached, the pressure is built up with the dome pressure regulator feeding only N2. At the same time, the heating of the whole piping periphery, the analytics, the process control system and the data acquisition must be prepared. Then an experiment can be started by dosing the reactants with the MFC. When the experiment is finished, the reactor is depressurized, inertized and most of the time cooled down to 180°C.
Catalysts are known to deactivate during operation ([
12]). The deactivation of the CZA methanol synthesis catalyst is addressed in several studies, e.g., [
13,
14,
15]. Catalyst deactivation is not the focus of this contribution, but must be taken into account. To quantify irreversible deactivation effects, a daily updated catalyst activity was measured prior to a new experimental run. This is done by applying a reference condition summarized in Table A5. The condition is kept until steady state is reached. Applying this procedure, a deactivation independent evaluation of the generated kinetic data is attempted. Additionally, due to the initial exposure to the same feed and thus the same reduction potential the catalyst is preconditioned with regard to reversible dynamic changes of the active sites. After that, the desired experiments were started.
The hydrogenation reactions (namely reaction 1 and 2) cause a reduction in gas volume, thus a volume contraction is defined as the ratio of inlet to outlet molar fraction of the inert nitrogen. Multiplying this ratio to the inlet volumetric flow rate
under norm conditions (273.15K and 1.01325bar) results in the norm outlet flow rate
:
Given the experimentally recorded molar fractions
of each component
i, the above calculated total outlet volumetric flow rate
and the ideal gas law concludes to the corresponding outlet molar flow rate
:
Engineering metrics as the carbon based yield of methanol can be calculated as shown in eq:yield MeOH C-based. This value results from the ratio of calculated outlet molar flow rate of methanol to the sum of input molar flow rates of all carbon species (i.e. CO and CO
2).
The value of the catalytic activity
a is determined via calculating the current methanol yield
with eq:yield MeOH C-based divided by the initial methanol yield
determined with the fresh catalyst of the corresponding charge, i.e. the stationary outlet result under reference condition after 1st reduction (Equation (7)).
The catalytic activity
a is used as an identical pre-factor for all three reaction rates as shown in Equation (8).
Thus, the assumption was made that the deactivation influences all three reactions Equations (1)–(3). in the same way and does not change drastically over the course of one specific experiment. Thus,
a is one fixed value for every experiment, ranging from 0.14 to 1 for the entire data used in this work and from 0.14 to 0.33 for the optimally designed experiments.
The whole set of experimental data used in this contribution is available under Kaps et al. (2025).
2.3. Mechanistic and Hybrid Modeling of the Methanol Synthesis
We base our approach upon the established kinetic model, capable to quantify the rates of the three key reactions proceeding during methanol synthesis, derived in [
5,
6,
7] (see F). This kinetic model is connected to the mass balances valid for an ideally mixed, isothermal, isobaric and continuously operated reactor model. The whole model consists of a set of differential-algebraic equations (DAE) of the form
Here denotes the time and denotes the state which consists of the mole fractions of the six chemical species , CO, , , , and an additional state which represents the distribution of the active centers of the catalyst. denotes a set of parameters (see G.1) and the inputs of the system, which consist of temperature, pressure, feed and flow. The matrix valued functions account for the gradient of the surface coverage of the individual species and combines the stoichiometric coefficients with the catalyst mass (constant for every experiment) in dependence of the states. denotes the activity level of the catalyst as described in Section 2.2.
An important feature of the kinetic model is the incorporation of dynamic changes of the catalyst.
captures the influence of the degree of reduction of the catalyst
on each of the reaction rates. Of special interest are the three scaling pre-factors
, which were previously modeled based on a heuristic
which only depends on the catalyst state
([
5,
6]). In this work, we propose to replace this heuristic with a fully connected feed-forward neural network to describe
, resulting in a hybrid model. Hybrid modeling aims at combining the advantages of two different modeling paradigms. First, the interpretability and data-efficiency of rigorous modeling based on first principles and specific domain knowledge, and second the flexibility of data-driven models such as neural networks, which are able to learn unknown functional relationships from data. In the context of differential equations, such models are commonly called universal differential equations ([
9]).
The hybrid DAE model can be written in its implicit form as
where
describes the parts of the model which are known from first principles and which may depend on physical parameters
. Second, a universal approximator, e.g., a feed-forward neural network,
with weights and biases
is used to model the unknown parts of the dynamics. The function
encodes the overall structure of the hybrid model, i.e., how known and unknown parts of the dynamics interact. Figure 4 shows the mechanistic model in the form of a block diagram, highlighting the catalyst impact
which we are replacing.
Figure 4.
Block diagram of the mechanistic model in the form of a DAE system. Our goal is to learn the impact of the catalyst state on the reaction rates r directly from the data. The triangle ▵ represents concatenation.
Figure 4.
Block diagram of the mechanistic model in the form of a DAE system. Our goal is to learn the impact of the catalyst state on the reaction rates r directly from the data. The triangle ▵ represents concatenation.
The feasibility of this approach applied already to methanol synthesis was shown in [
16], however based on synthetic data obtained via simulating the mechanistic model of [
5,
6,
7]. In this study, we focus on the training of hybrid models on experimental data.
2.5. Parameter Estimation
The parameter estimation was performed by solving the following optimization problem:
We used an average sample-wise (12a)
between the measurements
(outlet molar fractions) and predictions
x given by the weighted sum-of-squares error
We included only a subset of species, namely the carbon-based species with a high accuracy in the analytics and water as co-product, present often only in small amounts but very important for the kinetic model. The weights have been chosen to be
,
, to adjust the contribution of each species relative to its magnitude. () describes the dynamics of the system. Instead of modeling the domain (), we used the exponential transformation to ensure that a subset of parameters is positive to account for its physical meaning. The initial conditions and domains of the parameters have been set according to [
7], and can be seen in G.1. To solve (), we used
Optimization.jl ([
17]), a solver agnostic interface to several optimization routines. To fit the mechanistic model, we used the implementation of L-BFGS ([
18]) of
Optim.jl ([
19]). To investigate the impact of the experimental designs, once the parameter estimation problem is solved on the full training data with all experimental designs and once on the training data excluding the optimal experimental designs.
2.6. Hybrid Model Fitting
To find a suitable candidate for the hybrid model, we performed a simple hyperparameter search over different architectures by employing a full factorial design over the number and size of the hidden layers as well as their activation function in
Lux.jl (Pal (2023)). We adapted (12) by adding a L
1 norm regularization with regularization parameter
on the weights of the network to counteract overfitting. The options of this hyperparameter search are given in Table 1. The neural networks are modeled to take the full state
x as an input. Moreover, the output layer uses the softplus function
,
to ensure a positive output. Each candidate architecture has been trained five times with different random seeds used for initialization of the weights
for a total number of 500 iterations using
Adam ([
20]) from
Optimisers.jl and mini-batching of the data set. Next, a suitable candidate has been selected by accounting for both complexity of the network and conformity with the data. We selected the top-k candidates using the coefficient of determination on the training set and subsequently retrained these models using L-BFGS.
Table 1.
Hyperparameters for initial training of hybrid model. The function swish is given by , denotes the typical sigmoid function .
Table 1.
Hyperparameters for initial training of hybrid model. The function swish is given by , denotes the typical sigmoid function .
| Hyperparameter |
Hidden Layers |
Layer Width |
Activation |
| Options |
2, 3, 4 |
5, 10, 15 |
|
2.7. Optimal Experimental Design
To minimize the effort of experimental data collection, we want to collect as much information about the uncertain parameters as possible from each experimental run. Optimal experimental design (OED) is a well-established concept in Chemical Engineering ([
21,
22]) and a natural approach to come up with informative experiments for subsequent parameter estimation problems or model identification. See [
23,
24,
25,
26] for a general introduction into the topic. In [
27,
28] the concept was applied to dynamical systems given by parameterized ODE or DAE models. In this context, degrees of freedom are the decisions when to measure each observed variable
and how to stimulate the system in an optimal way. The problem of computing an optimal experimental design can thus be cast as an optimal control (OC) problem, see I.1 for detailed explanation. The goal is to optimize the time-dependent inputs
and the initial conditions
such that a suitable objective function of the variance-covariance matrix is minimized. This requires to augment the system using the forward sensitivity equations and the differential equation of the Fisher information matrix, adding states in the order of
.
In this work, we use OED in two ways. First, we use it to compute four informative experiments for estimating the physical parameters of the heuristic model. More specifically, these experiments each aim at the identification of different subsets of parameters, namely the parameters in the Arrhenius equations of the three elementary reactions and two rate constants in the kinetic equation for the state
. Second, in the case of a hybrid model, i.e., the parameters
q of the underlying model include both physical parameters
and weights and biases of a neural network
. In this context, the OED problem may also be solved aimed at generating data to train the embedded neural network on. However, problems may arise due to the usually large parameter space of the neural network. The quadratic dependence of the size of the augmented system of differential equations on the number of parameters as explained above leads to a high computational effort even for integration of the augmented system. Moreover, setting up the variational differential equations analytically may be prohibitively expensive. Another difficulty arises from the observation that weights of neural networks are not uniquely identifiable. Therefore, resulting Fisher information matrices in the underlying optimization problem are usually ill-conditioned and not amenable to optimization. However, as argued in [
10], one can often circumvent these problems by identifying a suitable subset of parameters
via a singular value decomposition of the Fisher information matrix. This way, the size of the system of differential equations is reduced and positive definiteness of the Fisher information matrix assured.
All OED problems were solved in a first-discretize-then-optimize single shooting approach on a time horizon
s with the A-Criterion as the objective function. The control functions
u include the inlet composition, i.e.,
and
as well as the inlet volumetric flow rate
and are discretized as piecewise constant values on an equidistant timegrid. For the experimental designs for the mechanistic model we chose a discretization parameter of
s, whereas for the hybrid model it was chosen as
s. The temperature
T is included as a control value, constant over time. The problems are further simplified by assuming that measurements are taken continuously, as explained in 2.1. Hence, we do not need to determine optimal measurement times. Furthermore, we need to include simple upper and lower bounds on the control functions and values, these are stated in Table 2. Also, general constraints on the inlet composition need to be fulfilled, requiring that the molar fractions of the incoming species add up to 100 percent at each point in time. Also, a minimum level of nitrogen needs to be present in the inlet due to required flushing of the stirrer magnet. The amount depends on the current volumetric flow rate and the pressure
p. Formally, these constraints read
Table 2.
Lower and upper bounds on control functions and control values for the optimal experimental design.
Table 2.
Lower and upper bounds on control functions and control values for the optimal experimental design.
| Variable |
Lower Bound |
Upper Bound |
Unit |
| T |
500.0 |
530.0 |
K |
|
0.5 |
4.0 |
NL / min |
|
0.0 |
40.0 |
% |
|
0.0 |
40.0 |
% |
|
0.0 |
75.0 |
% |
|
0.0 |
100.0 |
% |
The initial conditions of the molar fractions of the species and the initial distribution of active centers on the catalyst are given in Table 3. The pressure is fixed at
bar for all designs to simplify the optimization problem. Low reactant molar fractions in the feed are realized by compensation with nitrogen. This method also simplifies the experimental procedure and is in accordance to the assumed ideal gas behaviour. The problems are then solved via
Ipopt ([
29]). More information on the parameters considered in the designs and the obtained solutions are given in I.
Table 3.
Initial conditions assumed in all OED problems.
Table 3.
Initial conditions assumed in all OED problems.
| State |
Initial value |
|
0.00854 |
|
0.02494 |
|
0.22906 |
|
0.55062 |
|
0.00184 |
|
0.18499 |
|
0.7 |