An Approach to Generating Fuzzy Rules for a Fuzzy Controller Based on the Machine Learning Results Interpretation

2. Related Works

The management of complex technical systems requires an approach that ensures control stability, for example, based on the deterministic models. Intelligent components of a control system can identify behavior patterns of the object model based on analysis of the nonlinear and uncertain data with machine learning [3]. Component for predictive analytics plays a special role in control systems, because it allows to reduce the response time to emerging deviations. Quality of a control system is depends on the quality and volume of data, and form type and hyperparameters of a selected model as well.

Task of generating of a set of management rules is important and difficult because the quality of rules is influence to the quality of results of a control system, and an analytic needs to analyse the large volume of data to get rules with acceptable quality. The key feature of this task is the identification of features those influence on a quality of a control system result.

Various researchers use a different set of methods to generate a set of rules.

In paper [4] authors describe an approach to features extraction from a data.

In papers [5,6] described approaches for extraction a set of rules on data preparation stage based on a decision tree. Authors note that the approach based on a fuzzy rule base provides excellent opportunities for interpreting the results of data analysis. Authors also discusses a comparison of various methods for generating of a rule base based on a decision tree (ID3 algorithm), fuzzy decision tree, FUZZYDBD method. The authors propose an approach inspired by fuzzy decision tree approach based on ID3 algorithm that using information gain and Shannon’s entropy for feature selection criteria with fuzzification of the dataset.

In [7,8] authors consider the problem of the creation of control systems with fuzzy inference. Main problem is choosing the type of membership functions. The article also describes the algorithm for choosing the type of membership functions. The proposed approach is based on an algorithm to search for parameters of membership functions. Authors solve the problem of membership functions formation based on the statistical analysis of the features extracted from a training dataset. Researches focused on the original dataset as a basis for forming a high-quality classifying models.

Other researchers focused on the creation of fuzzy hierarchical systems [18,19,20,21]. In [9] consider the use of clusterisation based on fuzzy decision trees for multi-criteria decision making. Describing value intervals based on fuzzy sets allows to increase the flexibility of the system.

In the article [13] discussed the problem of constructing fuzzy decision trees, and the problem of choice of the type of membership functions.

In [10,11] authors describe approaches for creation of a control system based on a fuzzy rule bases generating with genetic evolutional algorithms. The main idea behind those approaches is finding an optimal solution to different problems based on analyzing large arrays of data.

Authors of [12,16,22] describe the usage of neuro-fuzzy networks to solve problems of nonlinearity of features of analysed objects in control tasks, for example, for energy storage systems.

The main problem of creating control systems based on fuzzy knowledge bases is the preparation of data and rule extraction. It is necessary to have a dataset with informative features to build a control system with acceptable quality [17]. Some methods require data labeling [14] or data preprocessing [7].

The options for improving the quality of a fuzzy control systems are:

• A high-quality result can be achieved by forming, normalizing, and optimizing a set of rules.
• It is necessary to select optimal membership functions and regulate their parameters to obtain high-quality results.

Rule mining approach can be based on rule classifier that was trained on existing labeled dataset [15].

Thus, existing approaches to the generation of rules for rule-based control systems cannot be used without deep knowledge of data analysis and statistics. Large amount of analytical work must be completed for high-quality tuning of hybrid models. Quality of the solution to the problem a rule generation and control problem itself is depend on expert opinion when need to choose the methods and their parameters and operating modes.

We define the main problem statement as the development of a method that allow to analyze the initial data in order to extract rules that have a high generalizing ability to identify patterns and operating modes of the control system [23]. Also, an approach to interpreting machine learning models can be used when developing such a method [24].

If non-deep machine learning methods cannot find patterns in the data, then it may be impossible to create a set of rules to achieve the required level of quality without deep expert knowledge.

3. Material and Methods

In Section 2 we presented an analysis of articles on the problem of generating fuzzy rules for FIS. That problem can be solved based on the interpretation and explanation of the results of machine learning models. Machine learning methods can find hidden patterns. Those patterns can be converted to a set of rules. The classical approach to the formation of a set of rules for a control system requires deep expert knowledge from an analytic.

In this paper, we propose an approach to generating fuzzy rules based on the analysis of a result of a machine learning model. The analysed model is created using a supervised learning algorithm based on decision trees.

3.1. Description of the Dataset

We used the dataset described in [25] to train the decision tree model. In [26], the authors used this dataset to create and evaluate a FIS-based control system. The dataset contains several tables. Each table contains measurements of the effect of the input parameters, aluminum oxide (Al2O3) and titanium dioxide (TiO2) dispersed in distilled water and ethylene glycol with 50:50 volumetric proportions on the density and viscosity parameters at different temperatures.

Thus, the input parameters from X are:

• Temperature ($temp$): 20-70 °C
• Al2O3 concentration ($al$): 0, 0.05, 0.3 vol %.
• TiO2 concentration ($ti$): 0, 0.05, 0.3 vol %.

Output parameters from Y are:

• Density ($density$).
• Viscosity ($viscosity$).

The Appendices Appendix A and Appendix B present the used datasets. Each dataset is divided into training and test sets.

3.2. Schema of the Proposed Approach

The Figure 1 shows the schema of the proposed approach to extracting fuzzy rules for constructing FIS based on the interpretation of decision tree results.

As you can see from the Figure 1, the input data is the training part of a dataset.

The CART algorithm [27] was chosen as the algorithm for training the model for creating a binary decision tree. The CART algorithm has the following advantages:

• There is no need to calculate and select various parameters to execute the algorithm.
• There is no need to pre-select the variables that will participate in the analysis to apply the algorithm. The variables are selected during model training based on the Gini index value.
• The algorithm handles outliers well. Separate tree branches are formed for data with outliers.
• High model training speed.

The major disadvantage of the CART algorithm is the low quality of the model for data with many dependencies between features. The solution to this problem is not covered in this article and will be solved in the future. The quality of the decision tree-based model is evaluated using the $R^2$ metric [27].

The decision tree model is formed after the CART algorithm execution. The decision tree model is the input data for the proposed approach to the generation of fuzzy rules. A set of fuzzy rules is generated as a result.

The resulting fuzzy rules are used to build FIS. The fuzzy rule $\tilde{r}$ can be represented as:

$$\tilde{r}=\langle Antecedent,Consequent\rangle ,$$

where $Antecedent=\{Ato{m}_1^A\circ Ato{m}_2^A\circ \dots \circ Ato{m}_n^A\}$ is the antecedent (condition) of the fuzzy rule; $Consequent=\{Ato{m}_1^C\circ Ato{m}_2^C\circ \dots \circ Ato{m}_m^C\}$ is the consequent (result) of the fuzzy rule; $Ato{m}_i^A,Ato{m}_j^A,$ are the i-th and j-th atoms of the antecedent and consequent, respectively; $\circ =\{or,and\}$ is an operator for connecting the atoms of the rule. $or$ and $and$ operators can be interpreted as functions of $min$ or $max$ depending on the fuzzy inference algorithm.

The operation of FIS is based on the principles of Zadeh’s fuzzy logic [28]. The operation of FIS can be described as a sequence of the following steps:

• Fuzzification of input values. The value of the input variable $x_i$ is assigned a set of linguistic terms of some fuzzy variable ${\tilde{x}}_i$ during fuzzification. Each fuzzy variable can be described as:

  $${\tilde{x}}_i=\langle N,T,U,F\rangle ,$$

  where N is the variable name: temperature, concentration; T is a set of linguistic terms: high temperature, medium temperature, low temperature, high concentration, medium concentration, low concentration; U is an range of values; F is a function for calculating the degree of membership of the input variable value to a certain linguistic term. The set of linguistic terms describes a subset of values of the fuzzy variable U. In this case, the value of the input variable is related to all linguistic terms with different membership degrees ${\mu }_t\left(x_i\right)\in \left[0,1\right]$.
• Aggregation. Truth degree ${\delta }^A$ of the rule antecedent is calculated at the aggregation stage:

  $${\delta }^A=F^A\left({\mu }_t\left(x_1\right),{\mu }_t\left(x_2\right),\dots ,{\mu }_t\left(x_n\right).\right)$$
  Each atom of the antecedent $Ato{m}_i^A$ of a fuzzy rule $\tilde{r}$ corresponds to a linguistic term $t_{ij}$ of some fuzzy variable ${\tilde{x}}_i$. Rule atoms are replaced by the values of the membership degree of the input variable $x_i$ to some linguistic term $t_{ij}$ during aggregation,. Then the function $F^A$ ($min$ or $max$) is applied. The implementation of the function $F^A$ is determined by the algorithm of fuzzy logical inference: Mamdani, Sugeno, Tsukamoto, etc.
• Activation. Truth degree ${\delta }^C$ of the consequent of the output variable ${\tilde{y}}_i$ is calculated at the activation stage,. In our case, the consequent always consists of one atom and has a weight coefficient equal to 1. Thus:

  $${\delta }^C={\delta }^A,{\mu }_t\left({\tilde{y}}_i\right)={\delta }^A.$$
• Accumulation. The membership function $F^{{\tilde{y}}_i}$ is formed for all output variables at the accumulation stage. The membership function is formed based on the max-union of the membership degrees of all linguistic terms of i-th fuzzy variable ${\tilde{y}}_i$:

  $$F^{{\tilde{y}}_i}=max({\mu }_t\left({\tilde{y}}_{1i}\right),{\mu }_t\left({\tilde{y}}_{2i}\right),\dots ,{\mu }_t\left({\tilde{y}}_{ki}\right)).$$
• Defuzzification. Numerical value for the fuzzy output variable ${\tilde{y}}_i$ is obtained based on the membership function $F^{{\tilde{y}}_i}$ at the defuzzification stage. In our case, the Centre of Gravity method is used:

  $$F^{Crisp}({\tilde{y}}_i,F^{{\tilde{y}}_i})={\displaystyle \frac{{\int }_{{\tilde{y}}_i^{min}}^{{\tilde{y}}_i^{max}}{\tilde{y}}_i\mu \left({\tilde{y}}_i\right)d{\tilde{y}}_i}{{\int }_{{\tilde{y}}_i^{min}}^{{\tilde{y}}_i^{max}}\mu \left({\tilde{y}}_i\right)d{\tilde{y}}_i}}.$$

The quality of FIS is evaluated on the test part of a dataser using the $R^2$ metric and compared with the value for the decision tree model.

The primary aim of this study is to confirm the Hypothesis 1.

Hypothesis 1.

It is possible to generate a set of fuzzy rules for constructing FIS based on the proposed algorithm. The quality of FIS must not be much worse in quality compared to the original decision tree model. Membership functions and labels of linguistic terms are generated automatically for all input and output variables. It is only necessary to specify the required number of terms: 3 or 5.

3.3. Description of the Approach to Generating Fuzzy Rules

In this section, we will consider the operation of the proposed approach to generating fuzzy rules using the example of constructing a FIS to determine the value of the output variable $density$ based on the values of the input variables $temp$, $al$, and $ti$. The data set is presented in the Table A1.

Decision tree $dt$ was created based on the training sample using the CART algorithm. Indicator $R_{dt}^2=0.9933$ was calculated based on the test data set for the decision tree $dt$. The resulting decision tree model was saved in a file for further use.

Step 1. Get a set of raw rules from the decision tree

Set of raw rules $r^{raw}$ is extracted from the previously created decision tree $dt$ at the first step of the proposed approach.

Formally, a rule extracted from a decision tree can be represented as:

$$r^{dt}=\langle Anteceden{t}^{dt},Consequen{t}^{dt}\rangle ,$$

where $Anteceden{t}^{dt}=\{Ato{m}_1^{dt},Ato{m}_2^{dt},\dots ,Ato{m}_i^{dt},Ato{m}_n^{dt}\}$ is the rule antecedent; $Ato{m}_i^{dt}=\langle x,type,value\rangle ,type\in [\le ,>]$ is the rule antecedent atom that describes the constraint of some input variable x with type $type$ and value $value$; $Consequen{t}^{dt}$ is the rule consequent that determines the value of the output variable $y_k$.

Extraction of a set of raw rules is performed based on the algorithm described in the work [30].

A raw rule is a rule that is extracted from a decision tree and contains an excessive number of conditions that may overlap, for example:

$$\begin{array}{cc}& \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\hfill \\ & \mathrm{and}(ti\le 0.025)\mathrm{and}(al\le 0.025)\mathrm{and}(temp>55.0)\hfill \\ & \mathrm{and}(temp>62.5)\to 1.033\hfill \end{array}$$

A set of raw rules $r^{raw}$ is presented in the Appendix C.

In the rule presented above, several conditions are imposed on the value of the input variables. These conditions must be simplified by performing normalization of the rules.

Step 2. Normalization of raw rules

Set of normalized rules $r^{norm}$ is formed from the set of raw rules $r^{raw}$ at the second step of the proposed approach. It is necessary to remove intersecting conditions from the raw rule $r_i^{raw}$ for all input variables to obtain a normalized rule $r_i^{norm}$. The normalization function can be represented as the Algorithm 1.

Algorithm 1 Rules normalization algorithm

function normalise($r^{raw}$, X)     $r^{norm}\leftarrow$ new list     for all $r_i^{raw}\in r^{raw}$ do         $Anteceden{t}_i^{norm}\leftarrow$ new list         for all $x_j\in X$ do            $A_j\leftarrow \{Ato{m}_{ijk}^{raw}\in Anteceden{t}_i|Ato{m}_{ijk}^{norm}.x=x_j\}$            ${Atom\le }_{ij}^{norm}\leftarrow max\left(\{Ato{m}_{ijk}^{norm}\in A_j|Ato{m}_{ijk}^{norm}.type=\le \}\right)$            ${Atom>}_{ij}^{norm}\leftarrow min\left(\{Ato{m}_{ijk}^{norm}\in A_j|Ato{m}_{ijk}^{norm}.type=>\}\right)$            $Anteceden{t}_i^{norm}.append({Atom\le }_{ij}^{norm},{Atom>}_{ij}^{norm})$         end for         $r_i^{norm}\leftarrow$ new Rule$\langle Anteceden{t}_i^{norm},Consequen{t}_i^{raw}\rangle$         $r^{norm}.append\left(r_i^{norm}\right)$     end for     return $r^{norm}$ end function

As you can see from the description of the Algorithm 1 algorithm, set of atoms $A_j$ is searched in the antecedent $Anteceden{t}_i^{raw}$ of each raw rule $r_i^{raw}\in r^{raw}$. The set $A_j$ contains the atoms of the rule antecedent that are associated with the input variable $x_j$. Then, the search for atoms with the ≤ type is performed and the atom ${Atom\le }_{ij}^{norm}$ with the maximum value of the parameter $variable$ is selected. Search for the atom ${Atom>}_{ij}^{norm}$ with the minimum value of the parameter $variable$ is performed among the atoms with the > type. The antecedent $Anteceden{t}_i^{norm}$ of the normalized rule $r_i^{norm}$ is formed based on the found atoms ${Atom\le }_{ij}^{norm}$ and ${Atom>}_{ij}^{norm}$. The consequent of the normalized rule $r_i^{norm}$ is the consequent of the raw rule $Consequen{t}_i^{raw}$.

An example of a normalized rule is shown below:

$$\begin{array}{cc}& \mathrm{Raw}\mathrm{rule}\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\hfill \\ & \mathrm{and}(ti\le 0.025)\mathrm{and}(al\le 0.025)\mathrm{and}(temp>55.0)\hfill \\ & \mathrm{and}(temp>62.5)\to 1.033\hfill \\ & \mathrm{Normalized}\mathrm{rule}\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\to 1.033\hfill \end{array}$$

The set of normalized rules $r^{norm}$ is presented in the Appendix D.

The Algorithm 1 forms a set of normalized rules $r^{norm}$. Normalized rules may have equivalent antecedents and different consequents. We call such rules as similar. Similar rules must be reduced to a single rule.

Step 3. Removing Similar Rules

The third step of the proposed method involves removing similar rules. Similar rules are rules with equivalent antecedents. Algorithm 2 formally represents the function of removing similar rules.

Algorithm 2 Algorithm for removing similar rules

function get_similar_rules($r_i$, r)     return $\{r_j\in r|r_j.Antecedent=r_i.Antecedent\}$ end function function group_rules(r)     $Antecedent\leftarrow r\left[0\right].Antecedent$     $Consequent\leftarrow \mathrm{avg}\left(\{r_i.Consequent\in r\}\right)$     return new Rule$\langle Antecedent,Consequent\rangle$ end function function delete_similar_rules($r^{norm}$)     $r^{sim}\leftarrow$ new list     for all $r_i^{norm}\in r^{norm}$ do         $r^{sim}.append(get\_similar\_rules(r_i^{norm},r^{norm}))$     end for     $\overline{r}\leftarrow \{r_i^{norm}\in r^{norm}|r_i^{norm}\notin r^{sim}\}$     $\widehat{r}\leftarrow$ new list     for all $r_i^{sim}\in r^{sim}$ do         $rules\leftarrow get\_similar\_rules(r_i^{sim},r^{sim})$         $\widehat{r}.append(create\_rule\left(rules\right))$     end for     return $\overline{r}\cup \widehat{r}$ end function

As you can see from the description of the Algorithm 2 algorithm, a list of similar rules $r^{sim}$ is formed at the first step. The get_similar_rules function is used to determine similar rules. Rules with equivalent antecedents are similar. Then a list $\overline{r}$ containing rules for which there are no similar rules in the original set $r^{norm}$ is formed. The reduced set $\widehat{r}$ is formed on the basis of the set of similar rules $r^{sim}$ by grouping the rules by equivalent antecedents. The consequents for the rules of the reduced set $\widehat{r}$ are calculated as the arithmetic mean of the consequents of the rules grouped by equivalent antecedents. The group_rules function is used to group the rules. The result of the algorithm is the union of the sets $\overline{r}$ and $\widehat{r}$ ($r^{norm}=\overline{r}\cup \widehat{r}$)

Let’s consider an example of the execution of the Algorithm 2. Before execution of the algorithm $\left|r^{norm}\right|=34$, after execution of the algorithm – $\left|r^{norm}\right|=24$. Example of a similar rules removing:

$$\begin{array}{cc}& \mathrm{Before}\mathrm{similar}\mathrm{rules}\mathrm{removing}\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\mathrm{and}(temp\le 55.0)\to 1.045\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\mathrm{and}(temp\le 55.0)\to 1.051\hfill \\ & \mathrm{After}\mathrm{similar}\mathrm{rules}\mathrm{removing}\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\mathrm{and}(temp\le 55.0)\to 1.048\hfill \end{array}$$

The set of normalized rules $r^{norm}$ after removing similar rules is presented in the Appendix E.

Next, it is necessary to move from the intervals of values of the atoms of the rules antecedents to specific values.

Step 4. Rules simplification

It is necessary to move from intervals in the atoms of the rule antecedents to specific values of variables after removing similar rules. Rules simplification is allow applying fuzzification to construct fuzzy rules.

Rules simplification can be represented as an Algorithm 3.

Algorithm 3 Rules simplification algorithm

function delete_similar_rules($r^{norm}$, $data$)     $r^{simp}\leftarrow$ new list     for all $r_i^{norm}\in r^{norm}$ do         $antecedent\leftarrow$ new list         for all $x_j\in X$ do            $n\leftarrow \{Ato{m}_{ijk}^{raw}\in Anteceden{t}_i|Ato{m}_{ijk}^{norm}.x=x_j,Ato{m}_{ijk}^{norm}.type=\le \}$            $m\leftarrow \{Ato{m}_{ijk}^{raw}\in Anteceden{t}_i|Ato{m}_{ijk}^{norm}.x=x_j,Ato{m}_{ijk}^{norm}.type=>\}$            $value\leftarrow 0$            if $n\ne \varnothing$ and $m\ne \varnothing$ then                $value\leftarrow \mathrm{avg}\left(n\left[0\right].value,m\left[0\right].value\right)$            end if            if $n\ne \varnothing$ and $m=\varnothing$ then                $value\leftarrow min\left(data\left[x_j\right]\right)$            end if            if $n=\varnothing$ and $m\ne \varnothing$ then                $value\leftarrow max\left(data\left[x_j\right]\right)$            end if            $atom\leftarrow$ new Atom$\langle x_j,=,value\rangle$            $antecedent.appned\left(atom\right)$         end for         $rule\leftarrow$ new Rule$\langle antecedent,Consequen{t}^i\rangle$         $r^{simp}.append\left(rule\right)$     end for     return $r^{simp}$ end function

As you can see from the description of the 3 algorithm, the left part of the interval n and the right part of the interval m are searched for each input variable $x_j$ in the antecedent atom $Anteceden{t}_i$ of the rule $r_i^{norm}$. If the antecedent of the rule $Anteceden{t}_i$ contains both parts of the interval, then the average value of the parameter $value$ of the atoms n and m is specified as the value of the simplified atom $atom$. If the antecedent of the rule $Anteceden{t}_i$ contains only the left part of the interval n, then the value of the new atom is set as the minimum value of the variable $x_j$ in the data set $data$. If the antecedent of the rule $Anteceden{t}_i$ contains only the right part of the interval m, then the value of the new atom is set as the maximum value of the variable $x_j$ in the data set $data$. New antecedent of the rule is formed based on the the process of atoms simplification, the consequent remains unchanged.

Figure 2 is schematically presented the Algorithm 3.

Let’s look at an example of simplified rules:

$$\begin{array}{cc}& \mathrm{Before}\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\to 1.033\hfill \\ & \mathrm{After}\text{:}\hfill \\ & \mathrm{if}(al=0.0)\mathrm{and}(ti=0.0)\mathrm{and}(temp=70)\to 1.033\hfill \\ & \hfill \\ & \mathrm{Before}\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\mathrm{and}(temp\le 62.5)\to 1.038\hfill \\ & \mathrm{After}\text{:}\hfill \\ & \mathrm{if}(al=0.0)\mathrm{and}(ti=0.0)\mathrm{and}(temp=47.5)\to 1.038\hfill \end{array}$$

The set of simplified rules $r^{simp}$ is presented in the Appendix F.

It is necessary to form fuzzy sets for the input variables and form a set of fuzzy rules based on atom fuzzification after simplifying the rules.

Step 5. Rule fuzzification

It is necessary to form fuzzy sets for the input and output variables to fuzzify the set of rules $r^{simp}$:

$$\begin{array}{c}\hfill F^{fuzz}:x_i\times n\to \widetilde{x_i},x_i\in X,{\tilde{x}}_i\in \tilde{X},\\ \hfill F^{fuzz}:y_j\times n\to \widetilde{y_j},y_j\in Y,{\tilde{y}}_j\in \tilde{Y}.\end{array}$$

(1)

The automatic method [32] of generation of fuzzy sets for crisp variables is used as an implementation of the $F^{fuzz}$ function. Thus, a corresponding fuzzy variable (${\tilde{x}}_i\in \tilde{X}$, ${\tilde{y}}_j\in \tilde{Y}$) is formed with the specified number of linguistic terms n for each variable ($x_i\in X$, $y_j\in Y$).

The Figure 3, Figure 4, Figure 5 and Figure 6 present the automatically generated fuzzy sets for the variables $al$, $ti$, $temp$ and $density$, respectively.

Next, the Algorithm 4 generates a set of fuzzy rules $r^{fuzz}$ based on a set of simplified rules $r^{simp}$.

Algorithm 4 Fuzzy rules generation algorithm

function get_fuzzy_rules($r^{simp}$)     $r^{fuzz}\leftarrow$ new list     for all $r_i^{simp}\in r^{simp}$ do         $Anteceden{t}^{fuzz}\leftarrow$ new list         for all $Ato{m}_{ij}\in Anteceden{t}_i^{simp}$ do            $m\leftarrow F^{fuzz}\left(Ato{m}_{ij}.x,Ato{m}_{ij}.variable\right),m_i\in m,m_i=\langle \widetilde{x_i},t_j,{\mu }_{t_j}\left(x_i\right)\rangle$            $Ato{m}_{ij}^{fuzz}\leftarrow m_i\in m|m_i.{\mu }_{t_j}\left(x_i\right)\to max$         end for         $Atom\leftarrow Consequen{t}^{simp}\left[0\right]$         $m\leftarrow F^{fuzz}\left(Atom.x,Atom.variable\right)$         $Consequen{t}^{fuzz}\leftarrow m_i\in m|m_i.{\mu }_{t_j}\left(x_i\right)\to max$         $r_i^{fuzz}\leftarrow$ new Rule$\langle Anteceden{t}^{fuzz},Consequen{t}^{fuzz}\rangle$         $r^{fuzz}.append\left(r_i^{fuzz}\right)$     end for     return $r^{fuzz}$ end function

As you can see from the description of the 4 algorithm, fuzzification function $F^{fuzz}$ is executed for each atom $Ato{m}_{ij}$ of the antecedent of the crisp rule $r_i^{simp}$. m set is formed after fuzzification. Each element of the set $m_i$ contains the membership degree ${\mu }_{t_j}\left(Atom.x\right)$ of the crisp variable $Atom.x$ to the linguistic term $t_j$ of the fuzzy variable ${\tilde{x}}_i$. An atom of the fuzzy rule $Ato{m}_{ij}^{fuzz}$ is formed using the function max based on the set m. Thus, the atom of the fuzzy rule $Ato{m}_{ij}^{fuzz}$ contains a reference to the fuzzy variable ${\tilde{x}}_i$, as well as the degree of membership ${\mu }_{t_j}\left(Atom.x\right)$ in the linguistic term $t_j$. Atom of a consequent is formed similarly to the atoms of an antecedent.

Let’s consider an example of fuzzy rules:

$$\begin{array}{cc}& \mathrm{Before}\text{:}\hfill \\ & \mathrm{if}(al=0.0)\mathrm{and}(ti=0.0)\mathrm{and}(temp=70)\to 1.033\hfill \\ & \mathrm{After}\text{:}\hfill \\ & \mathrm{if}\left(al\mathrm{is}low\right)\mathrm{and}\left(ti\mathrm{is}low\right)\mathrm{and}\left(temp\mathrm{is}high\right)\to \left(density\mathrm{is}lower\right)\hfill \end{array}$$

Rules with similar antecedents and different consequents may be formed after 4 algorithm executing. Algorithm 2 is used to delete similar fuzzy rules. This algorithm was adapted to work with fuzzy rules. A special function group_fuzzy_rules (Algortihm 5) was developed to group fuzzy rules.

Algorithm 5 Group fuzzy rules algorithm

function group_fuzzy_rules(r)     $r^{min}\leftarrow r_i\in r|{\sum }_{j=1}^{n}Ato{m}_{ij}.{\mu }_t\left(x\right)\to min,Ato{m}_{ij}\in Anteceden{t}_i$     return $r^{min}$ end function

As you can see from the description of the 5 algorithm, the group_fuzzy_rules function remains only one of the similar rules $r^{min}\in r^{fuzz}$ in which the antecedent atoms have the minimum total value of membership degrees ${\sum }_{j=1}^{n}Ato{m}_{ij}.{\mu }_t\left(x\right)\to min$.

The set of simplified rules $r^{fuzz}$ is presented in the Appendix G. The number of rules in the set before fuzzification $r^{simp}=24$, and after fuzzification $\left|r^{fuzz}\right|=15$.

It becomes possible to perform fuzzy inference to get the value of the output variables Y based on the input variables X after obtaining the set of fuzzy rules.

Step 6. Fuzzy Inference

Fuzzy inference allows to get the value of crisp output variables Y based on crisp input variables X. Fuzzy rules are used in the inference process to describe an expert knowledge as the functional dependence $F:X\to Y$.

For example, for input variables $al=0$, $ti=0$, and $temp=25$:

• Fuzzification:
  • ${\mu }_{low}\left(al\right)=1.0$, ${\mu }_{average}\left(al\right)=0.0$, ${\mu }_{high}\left(al\right)=0.0$;
  • ${\mu }_{low}\left(ti\right)=1.0$, ${\mu }_{average}\left(ti\right)=0.0$, ${\mu }_{high}\left(ti\right)=0.0$;
  • ${\mu }_{low}\left(temp\right)=0.8$, ${\mu }_{average}\left(temp\right)=0.2$, ${\mu }_{high}\left(temp\right)=0.0$.
• Aggregation and activation:
  • For rule:
    $\mathrm{if}\left(al\mathrm{is}low\right)\mathrm{and}\left(ti\mathrm{is}low\right)\mathrm{and}\left(temp\mathrm{is}average\right)\to \left(density\mathrm{is}lower\right)$
    ${\delta }_1^A=min\{1.0,1.0,0.2\}=0.2$
    ${\delta }_1^C={\delta }_1^A=0.2$;
  • For rule:
    $\mathrm{if}\left(al\mathrm{is}low\right)\mathrm{and}\left(ti\mathrm{is}low\right)\mathrm{and}\left(temp\mathrm{is}low\right)\to \left(density\mathrm{is}low\right)$
    ${\delta }_2^A=min\{1.0,1.0,0.8\}=0.8$
    ${\delta }_2^C={\delta }_2^A=0.8$;
  • For rule:
    $\mathrm{if}\left(al\mathrm{is}high\right)\mathrm{and}\left(temp\mathrm{is}average\right)\to \left(density\mathrm{is}high\right)$
    ${\delta }_3^A=min\{0.0,0.0,0.2\}=0.0$
    ${\delta }_3^C={\delta }_3^A=0.0$, etc.
• Accumulation. Figure 7 represents the accumulation result.
• Defuzzification. $density=1.076,density\in Y$.

3.4. Rules Clustering

Rule clustering allows to grouping the rules based on the parameters of the rule antecedent atoms. The groups of rules allow an expert to evaluate the rules and set the hyperparameters for the proposed fuzzy rule generation method. The proposed rule clustering algorithm can be used for crisp and fuzzy rules.

For example, such groups in the A1 data are rows 1–9, 10–15, etc. It is necessary to specify a set of input variables to combine rules into groups. Atoms of a rule antecedent are selected based on selected input variables. For example, clustering can be performed in the A1 data set based on atoms with the $al$ and $ti$ variables. Atoms with the $temp$ variable can be ignored, because the variable value is repeated in each group of data rows. The set of variables is a hyperparameter of the rule clustering algorithm. Only atoms that are associated with the variables $al$ or $ti$ (the parameter of the atom x) are used in this example. The variable $temp$ was excluded.

It is necessary to vectorize the rules to perform clustering. The algorithm for generating a unique list of atoms is presented in 6.

Algorithm 6 Algorithm for generating a unique list of atoms

function get_unique_atoms($r^{norm}$, $X^{ex}$)     $atoms\leftarrow$ new set     for all $r_i^{norm}\in r^{norm}$ do         for all $a_j\in r_i^{norm}.Antecedent$ do            if $X^{ex}\ne \varnothing$ and $a_j.variable\in X^{ex}$ then                break            end if            $atoms.insert\left(a_j\right)$         end for     end for     return $atoms$ end function

As you can see from the description of the Algorithm 6, the result of the algorithm is a set of unique atoms $atoms$ extracted from the set of rules $r^{norm}$. Only atoms with a parameter $variable$ whose value is not contained in the set of excluded variables $X^{ex}$ are added to the set $atoms$.

The following set of unique atoms is formed:

$$\begin{array}{c}\hfill atoms=\left\{\right(al\le 0.175),(al>0.025),(al>0.175),(ti\le 0.175),(ti>0.025),(ti>0.175\left)\right\}.\end{array}$$

The $atoms$ set is used in the vectorization process as a binary mask. For example, for the rule $r_1^{norm}\in r^{norm}$ the resulting vector ${\overline{v}}_1^{norm}$ is:

$$\begin{array}{cc}& r_1^{norm}=\mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\to 1.033\hfill \\ & {\overline{v}}_1^{norm}=\langle 1,0,0,1,0,0\rangle .\hfill \end{array}$$

Process of automatic selection of clusters number is performed after vectorization. The minimum value of the clusters number is $k^{min}=2$. The maximum value of the clusters number can be specified manually by the user or it can be calculated as $k^{max}=\sqrt{\left|r^{norm}\right|}+1$. Automatic selection of clusters number is based on the value of the silhouette coefficient s [31]:

$$s={\displaystyle \frac{b-a}{max\left(a,b\right)}},$$

where a is the mean intra-cluster distance, b is the distance between a sample and the nearest cluster that the sample is not a part of. The best value is 1 and the worst value is -1. Values near 0 indicate overlapping clusters. Negative values indicate that a sample has been assigned to the wrong cluster, as a different cluster is more similar.

KMeans algorithm is used for clustering. n iterations of the clustering algorithm are sequentially performed for each $k_i\in 〚k^{min},k^{max}〛$. Silhouette coefficient $s_i$ is calculated (see Figure 8) for each iteration $k_i$ and the minimum value of the clusters number ($k_i$) with the maximum of the $s_i$ value is selected. Thus, the best value of $s_i$ was obtained at iteration $i=4$ when splitting into five clusters.

Result of the rules clustering is:

$$\begin{array}{cc}& \mathrm{Cluster}1\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp>32.5)\to 1.033\hfill \\ & \dots \hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(temp\le 32.5)\to 1.062\hfill \\ & \dots \hfill \\ & \mathrm{Cluster}3\text{:}\hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(ti>0.025)\mathrm{and}(temp>32.5)\to 1.056\hfill \\ & \dots \hfill \\ & \mathrm{if}(al\le 0.175)\mathrm{and}(ti\le 0.175)\mathrm{and}(ti>0.025)\mathrm{and}(temp\le 32.5)\to 1.091\hfill \\ & \dots \hfill \end{array}$$

Full result of the rules clustering is presented in the Appendix H.

4. Experiments

We develop an application to test the hypothesis 1. The main parameters of the environment for the developed application include:

• Programming language: Python.
• Python interpreter version: 3.12.
• Libraries:
  • Machine learning library (decision tree and KMeans clustering): scikit-learn 1.5.2;
  • Data manipulation libraries: numpy 2.1.0 and pandas 2.2.2;
  • Fuzzy inference library: scikit-fuzzy 0.5.0;
  • Plotting library: matplotlib 3.9.2;
  • Additional dependency for the scikit-fuzzy library: networkx 3.4.2.

Decision tree models were created for the output variables $density$ and $viscosity$ based on the training set of the A1 and A2 datasets. The following variables $al$, $ti$, and $temp$ were used as input variables in both experiments.

The following $R^2$ metric values were calculated for the resulting decision tree models based on the test set of the A1 and A2 datasets:

• $R_{density}^2=0.99$;
• $R_{viscosity}^2=0.83$.

Algorithm was extracted the following raw rules from the resulting decision trees:

• $\left|r_{density}^{raw}\right|=34$;
• $\left|r_{viscosity}^{raw}\right|=35$;

The following rules were obtained after executing the algorithms for normalization and removal of similar rules:

• $\left|r_{density}^{norm}\right|=24$;
• $\left|r_{viscosity}^{norm}\right|=26$;

Then, the proposed algorithm generated the following fuzzy rules:

• $\left|r_{density}^{fuzz}\right|=15$;
• $\left|r_{viscosity}^{fuzz}\right|=19$.

Figure 3, Figure 4, Figure 5 and Figure 6 represents fuzzy sets for the $density$ output variable.

Figure 3, Figure 4, Figure 9 and Figure 10 represents fuzzy sets for the$viscosity$ output variable.

Automf algorithm from the scikit-fuzzy library [32] was used to generate all fuzzy sets. The number of linguistic terms is a hyperparameter of the proposed approach. We selected its value during the experiments.

The Table 1 contains the FIS results. The column Real contains real data, and the column Inferred is a result of fuzzy inference.

The following values of the $R^2$ metric were calculated for the FIS based on the test set of the Appendices A1 and A2 datasets:

• ${\tilde{R}}_{density}^2=0.97$;
• ${\tilde{R}}_{viscosity}^2=0.81$.

Let’s calculate the difference between the $R^2$ indicators for decision tree models and the FISs:

• ${\Delta }_{density}=R_{density}^2-{\tilde{R}}_{density}^2=0,014$;
• ${\Delta }_{viscosity}=R_{viscosity}^2-{\tilde{R}}_{viscosity}^2=0,025$.

The average difference in the $R^2$ metric is about 2 %. The hypothesis is proven.

5. Conclusions

We consider an approach to solving control problems based on fuzzy logic. This approach allow to develop decision support systems for various application areas. The article considering the example of generating a set of fuzzy rules based on the interpretation of a decision tree model. The limitations of the proposed approach is the ability to work with dataset on which the CART algorithm shows an acceptable result.

Future work plans include:

• Development of an approach to generating a set of fuzzy rules based on the interpretation of other machine learning algorithms.
• Development of a method for generating fuzzy sets, considering the specifics of the subject area to improve the FIS quality.