A Soft Sensor Model Based on ISOA-GPR Weighted Ensemble Learning for Marine Lysozyme Fermentation Process

2.1. Data subsets Construction Strategy

Using the distribution of marine lysozyme fermentation process data, we propose an improved density peak clustering algorithm (ADPC) that evaluates the similarity between data in terms of the proximity between data samples. Density peak clustering (DPC) is a typical methodology founded on density clustering [14]. The cluster center is assumed to have a more significant local density and a greater relative distance ${\delta }_i$ from other cluster centers than other data points. The algorithm also requires that each data point relied on for classification has two feature values, local density ${\rho }_i$ and relative distance ${\delta }_i$.

For the sample set $R$, the local density ${\rho }_i$ of data $x_i$ is

$${\rho }_i={\displaystyle {\sum }_{i\ne j}\exp \left(-{\left(\frac{dis{t}_{ij}}{dis{t}_c}\right)}^2\right)}$$

(1)

Where $dis{t}_{ij}$ is the distance between data $x_i$ and $x_j$, and $dis{t}_c$ is the truncation distance.

This research employs a declining trend inscription adaptive cluster center acquisition approach to increase cluster center selection accuracy. Because the DPC algorithm's clustering center has a higher local density and relative distance than other data points, the logarithmic function was picked to accentuate the disparity between the clustering center and other data points. After arranging the acquired choice parameters in descending order, the declining trend of ${\gamma }_i$ values is determined as ${\gamma }_i^{\ast }$.

Define a decision parameter ${\gamma }_i$ that combines local density and relative distance:

$${\gamma }_i={\rho }_i\times \lg \left({\delta }_i\right)$$

(2)

$${\gamma }_i^{\ast }=\frac{{\gamma }_{i-1}-{\gamma }_i}{{\gamma }_i-{\gamma }_{i+1}}$$

(3)

Where ${\gamma }_i$ represents the current $\gamma$ value and ${\gamma }_{i-1}$ ${\gamma }_{i+1}$ represent the $\gamma$ values at the preceding and subsequent times, respectively.

The method was applied to the marine lysozyme data samples, and the distribution of decision parameters and the decreasing trend of decision parameters were obtained, as shown in Figure 1 and Figure 2, respectively.

2.2. Sub-Model Construction

The real marine lysozyme fermentation process exhibits apparent non-linear properties, a tiny data sample size, and challenging offline extraction. The Gaussian process regression method was chosen to establish a sub-prediction model for marine lysozyme fermentation in this paper [15]. For Gaussian process regression models, the choice of hyperparameters substantially affects the prediction model's precision. Traditional parameter selection methods rely heavily on experience and trial and error; regression accuracy and calculation speed are not guaranteed. In order to generate a sub-model with a better prediction effect, this paper uses the Improved Seagull Optimization Algorithm (ISOA) for online optimization and adjustment of hyperparameters.

2.3. Improved Seagull Optimization Algorithm

The seagull optimization algorithm (SOA) is an intelligent algorithm that simulates individual seagull flocks in nature and seeks to perform iterative optimization search in the solution space by employing the long-distance migration and spiral attack behavior of individual seagulls with the change of seasons [16].

In the conventional seagull optimization algorithm, the inertia weight decreases linearly as the number of iterations increases. Even though the repetition speed is faster, it is easy to cause the population variety to fall with each iteration. There is also a problem with weak global search ability in the early stage and poor local mining ability in the later stage. So this paper proposes a non-linear change in the inertia weight updating strategy. The specific expression is as follows:

$$A=-f_C\times \tan \left(\frac{t}{Ma{x}_{iteration}}\times \frac{\pi }{4}-\frac{\pi }{4}\right)$$

(4)

Where $t$ is the current number of iterations, $Ma{x}_{iteration}$ is the maximum number of iterations, and $f_C$ is a constant whose initial value is set to 2.

In the early iterations of the improved seagull optimization algorithm, the inertia weight $A$ decreases abruptly to maintain population diversity while enhancing its global search capability; in the later stages of execution, the inertia weight $A$ decreases gradually to increase the local search capability while ensuring that the algorithm is not easily trapped in a local optimum. Therefore, the optimal adjustment for the hyperparameters with the improved seagull optimization algorithm will undoubtedly result in a more accurate soft sensor model.

2.4. Submodel Selection and Fusion Strategy

This paper finds the weighted "center of mass" $Z_m$ that best represents the whole data subset and describes the relationship between the sub-model and the test sample by the degree of association between $Z_m$ and the test sample so that the right sub-model can be selected for integration weighting. The correlation coefficient between the test sequences and the weighted "center of mass" of the local subset was analyzed using an improved gray correlation algorithm that more accurately reflects the fluctuation between the marine lysozyme fermentation data sequences to determine their degree of correlation. Given a sample subset of marine lysozyme fermentation process data $\mathrm{r}=\left\{x_i;i=1,2,\dots ,{\mathrm{n}}_{\mathrm{m}}\right\}$, where $x_i\in {\mathrm{R}}^{\mathrm{d}},\mathrm{n}$ is the number of samples in each subgroup and $d$ is the feature variable's dimensionality. Let the reference sequence be $x_0=\left\{x_0\left(1\right),x_0\left(2\right),\dots ,x_0\left(d\right)\right\}$ and calculate the gray correlation coefficient:

$${\varsigma }_i\left(k\right)=\frac{\underset{n}{\min } \underset{k}{\min }\left|\Delta \right|+\rho \underset{n}{\max } \underset{k}{\max }\left|\Delta \right|}{\left|\Delta \right|+\rho \underset{n}{\max } \underset{k}{\max }\left|\Delta \right|},\rho \in [0,1]$$

(5)

Where $\Delta =\sqrt{{\left(x_0\left(k\right)-\overline{x_0}\right)}^2}-\sqrt{{\left(x_i\left(k\right)-\overline{x_i}\right)}^2},\left(k=1,2,\dots ,d\right)$ $\overline{x_i}=\frac{1}{d}{\displaystyle \sum _{k=1}^d{x}_i\left(k\right)}$ $\rho$ indicates the resolution coefficient, which is taken as 0.5.

The correlation between the reference and comparison sequences is calculated as follows:

$${\phi }_i=\frac{1}{d}{{\displaystyle \sum \varsigma }}_i\left(d\right)$$

(6)

So that each sample of the local sample subset is used as a reference sequence and the remaining samples are comparison sequences, the correlation matrix of the generated samples is calculated as follows:

$$\varphi =\left(\begin{array}{cccc}1& {\phi }_{12}& \cdots & {\phi }_{1n}\\ {\phi }_{21}& 1& \cdots & {\phi }_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ {\phi }_{n1}& {\phi }_{n2}& \cdots & 1\end{array}\right)$$

(7)

The sample with the strongest correlation with all subsets is picked as the data set's initial center of mass, and its correlation coefficients with other samples in that subset are reported to generate the correlation coefficient matrix:

$$\psi =\left(\begin{array}{cccc}{\zeta }_1\left(1\right)& {\zeta }_1\left(2\right)& \cdots & {\zeta }_1\left(k\right)\\ {\zeta }_2\left(1\right)& {\zeta }_2\left(2\right)& \cdots & {\zeta }_2\left(k\right)\\ ⋮& ⋮& \ddots & ⋮\\ {\zeta }_n\left(1\right)& {\zeta }_n\left(2\right)& \cdots & {\zeta }_n\left(k\right)\end{array}\right)$$

(8)

In this paper, we present information entropy weighting to characterize the degree of variation for each feature variable under the correlation coefficient matrix, assign objective weights to the feature variables, and derive an objective subset of the "center of mass." In general, the lower a feature variable's information entropy, the larger its degree of variation and the higher its given weight. Conversely, when information entropy increases, the relevance of feature variables decreases, and weights decrease. The characteristic weight of the $j$nd characteristic variable of the $i$th sample is calculated as:

$$P_{ij}=\frac{{\zeta }_i\left(j\right)}{{\Sigma }_{i=1}^n{\zeta }_i\left(j\right)}$$

(9)

Entropy value of the $j$ characteristic variable：

$$\{\begin{array}{l}{e}_j=\raisebox{1ex}{$-1$}\!\left/ \!\raisebox{-1ex}{$\ln \left(n\right)$}\right.{\displaystyle {\sum }_{i=1}^n{P}_{ij}\ast \ln \left(P_{ij}\right),P_{ij}\ne 0}\hfill \\ \underset{P_{ij}\to 0}{\lim }{P}_{ij}\ast \ln \left(P_{ij}\right)=0,P_{ij}=0\hfill \end{array}$$

(10)

Then, the weights of each characteristic variable are in the correlation system matrix.

$$w_j=\frac{1-e_j}{d-{\displaystyle \sum _{i=1}^d{e}_j}}$$

(11)

The previous computation produces the weighted center-of-mass $Z_m=w_j{Z}^{\ast }$ of the m sample subsets of the fermentation process, assuming that the enhanced density peak clustering approach correctly collects m local sample subsets. We obtained the $x^{\ast }$ correlation set $\omega =\left[{\omega }_1,{\omega }_2,{\omega }_3,\dots ,{\omega }_{\mathrm{m}}\right]$using the fermentation test sample as the reference sequence and the m subsets of "center of mass" $Z_m$ as the comparison sequence. We kept the ISOA-GPR sub-model corresponding to a correlation degree greater than or equal ω*. Its corresponding fermentation process sub-model prediction result is $y_{pre}=\left[y_{pre1},y_{pre2},y_{pre3},\dots ,y_{pre\eta }\right],\eta \in \left[1,m\right]$, so the final prediction result of gray correlation weighted integration is:

$$y_{prediction}=\frac{{\omega }_1}{{\displaystyle \sum {\omega }_{\eta }}}{y}_{pre1}+\frac{{\omega }_2}{{\displaystyle \sum {\omega }_{\eta }}}{y}_{pre2}+\frac{{\omega }_3}{{\displaystyle \sum {\omega }_{\eta }}}{y}_{pre3}+\cdots \frac{{\omega }_{\eta }}{{\displaystyle \sum {\omega }_{\eta }}}{y}_{pre\eta }$$

(12)