Optimizing Attribute Reduction in Multi-granularity Data through a Hybrid Supervised-Unsupervised Model

1. Introduction

In this era of information explosion, data is growing exponentially in both dimension and volume, which leads to the attributes of data becoming redundant and vague. How to find valuable information from massive data has become challenging. Rough set theory, introduced by Pawlak [1] in 1982 as a simple and efficient method for data mining, can deal with fuzzy, incomplete, and inaccurate data.

The traditional model of rough sets mainly focuses on describing the uncertainty and fuzziness of data through binary relations. In recent years, multi-granularity rough set models have been proposed to fully mine the multiple granularity levels of target information, extending the traditional single binary relation to multiple binary relations, with the work of Qian et al. [5] being representative. This model has provided a new solution for rough set theory in dealing with distributed data and multi-granularity data. Afterward, researchers continuously improve Qian’s multi-granularity rough set model. Some of the improvements combine multi-granularity rough sets with decision-theoretic rough sets to form a multi-granularity decision-theoretic rough set model [6]. In addition, there is research combining multi-granularity rough sets with the three-way decision model, proposing a multi-granularity three-way decision model [7]. Targeting the granulation of attributes and attribute values, Xu proposed an improved multi-granularity rough set model [8]. To expand the applicability of multi-granularity rough sets, Lin et al. integrated the neighborhood relation into the multi-granularity rough set model, proposing the neighborhood multi-granularity rough set. The introduction of this model has made the multi-granularity rough set research branch a hot topic of study [9]. These rough set models can effectively reduce data dimensionality, achieved by attribute reduction.

Attribute reduction can be achieved through supervised or unsupervised constraints, and research on constraints from both supervised and unsupervised perspectives has been extensively explored. [10]. Specifically, some studies propose attribute reduction constraints based on measures from only one perspective, using these constraints to find qualified reductions. For instance, Jiang et al. [11] and Yuan et al. [12] concentrated on attribute reduction through the lens of supervised information granulation and related supervised metrics, respectively; meanwhile, Yang et al. [13] proposed a concept known as fuzzy complementary entropy for attribute reduction within an unsupervised model. However, whether considering supervised measures or unsupervised measures, single-perspective based measures exhibit inherent constraints. Firstly, measures relying on a single perspective may overlook the multifaceted evaluation of data, leading to the neglect of some important attributes. This is because when only one fixed measure is used for the attribute reduction of data, the importance of each attribute is judged only based on its criterion. However, if other measures are needed for evaluation, then relying only on that criterion may no longer yield accurate results. Secondly, relying only on a single-perspective measure may not fully capture the characteristics of data under complex conditions, resulting in the selection of attributes that are neither accurate nor complete. For instance, if conditional entropy is used as a measure to evaluate attributes [17], the derived reduction may only possess the single feature required for evaluation, without fully considering other types of uncertainty features and learning capabilities.

In solve the limitations of the attribute reduction mentioned above, this paper introduces a new measure that merges both supervised and unsupervised perspectives, leading to a novel rough set model. The model proposed in this paper has the following advantages: (1) It integrates multi-granularity and neighborhood rough sets, making the model more adaptable to data features at different levels; (2) For attribute sets of different granularities, it introduces a fusion strategy, selecting the optimal granularity level or attribute set according to the needs of different tasks and datasets, which can be flexibly adjusted based on specific circumstances.

The rest of this paper is organized as follows. Section 2 reviews related basic concepts. Section 3 provides a detailed introduction to the basic framework and algorithm design of the proposed method. In Section 4, the accuracy of our method is calculated and discussed through experiments. Finally, Section 5 concludes this paper and depicts some future works.

2. Preliminaries

2.1. Neighborhood Rough Sets

Neighborhood rough sets were proposed by Hu et al. as an improvement over traditional rough sets [18]. The key distinction lies in that neighborhood rough sets are established on the basis of neighborhood relations, as opposed to relations of indiscernibility. Hence, the neighborhood rough set model is capable of processing both discrete and continuous data. [36] Moreover, the partitioning of neighborhoods granulates the sample space, which can reflect the discriminative power of different attributes on the samples.

Within the framework of rough set theory, a decision system is characterized by a tuple, represented by $DS=(U,AT)$, where U denotes a finite collection of samples and $AT$ encompasses a suite of conditional attributes, including a decision attribute d [19]. The attribute d captures the sample labels. For every x in U and every a in $AT$, $a\left(x\right)$ signifies the value of x for the conditional attribute a, and $d\left(x\right)$ represents the label of x. Utilizing d, one can derive an equivalence relation on U:

$$IND\left(d\right)=\left\{\right(x,y)\in U\times U:d(x)=d(y\left)\right\}$$

(1)

Pursuant to $IND\left(d\right)$, it leads to a division of $U/IND\left(d\right)=X_1,X_2,\dots ,X_q(q\ge 2)$. Each $X_k$ within $U/IND\left(d\right)$ is recognized as the k-th decision category. Notably, the decision category that includes the sample x can be similarly referred to as ${\left[x\right]}_d$.

In rough set methods, binary relations are often used for information granulation, among which neighborhood relations, as one of the most effective binary relations, have received extensive attention. The formation of neighborhood relations is as follows:

$$N_{\delta }^A=\{(x,y)\in U\times U:{\delta }_A(x,y)\le \delta \}$$

(2)

where $r_A$ is a distance function regarding $A\subseteq AT$, $r\ge 0$ is a radius.

$$r_A=\sum _{a\in A}{(a\left(x\right)-a\left(y\right))}^2$$

(3)

Based on $IND\left(d\right)$, a segmentation of $U/IND\left(d\right)=X_1,X_2,...,X_q(q\ge 2)$ can be initiated. For every $X_k$ within $U/IND\left(d\right)$, it is identified as the k-th decision group. In particular, the decision group that encompasses sample x may also be represented as ${\left[x\right]}_d$.

In alignment with Equation (2), the vicinity of a sample x is established as follows:

$${\delta }_A\left(x\right)=\{y\in U:r_A(x,y)\le \delta \}$$

(4)

From the perspective of granular computing [21,22], both $IND\left(d\right)$ and $N_{\delta A}$ are derivations of information granules. The most significant difference between these two types of information granules lies in their intrinsic mechanisms, i.e., the binary relations used. Based on the outcomes of these information granules, the concepts of lower and upper approximations within the context of neighborhood rough sets, as the fundamental units, were also proposed by Cheng et al.

2.2. Multi-Granularity Rough Sets

For multi-granularity rough sets [23,24,26], given $DS=(U,AT)$, where $AT=\left\{A_k\right|k\in \{1,2,\dots ,m\}\}$ is a set of attributes, and the family of attribute subsets on $AT$ is represented by $\{A_1,A_2,\dots ,A_m\}$.

${\left[x\right]}_{A_i}$ is an equivalence class of x under $A_i$, for any $X\subseteq U$, the optimistic multi-granularity lower and upper approximations of $A_i$ with respect to X are defined as follows:

$$\underline{\sum _{\begin{array}{c}i=1\end{array}}^m{A}_i^O}\left(x\right)=\{x\in U\mid {\left[x\right]}_{A_1}\subseteq X\vee {\left[x\right]}_{A_2}\subseteq X\vee \dots \vee {\left[x\right]}_{A_m}\subseteq X\},$$

(5)

$$\overline{\sum _{\begin{array}{c}i=1\end{array}}^m{A}_i^O}\left(x\right)={\left(\underline{\sum _{\begin{array}{c}i=1\end{array}}^m{A}_i^O}\left(x^c\right)\right)}^c$$

(6)

If $\underline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^O}\left(x\right)\ne \overline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^O}\left(x\right)$, then $\underline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^O}\left(x\right)$ and $\overline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^O}\left(x\right)$ are called optimistic multi-granularity rough sets.

Given $DS=(U,AT)$, where $AT=\left\{A_k\right|k\in \{1,2,\dots ,m\}\}$ is a set of attributes, and the family of attribute subsets on $AT$ is represented by $\{A_1,A_2,\dots ,A_m\}$.

${\left[x\right]}_{A_i}$ is an equivalence class of x under $A_i$, for any $X\subseteq U$, the pessimistic multi-granularity [27] lower and upper approximations of $A_i$ with respect to X are defined as follows:

$$\underline{\sum _{\begin{array}{c}i=1\end{array}}^m{A}_i^P}\left(x\right)=\{x\in U\mid {\left[x\right]}_{A_1}\subseteq X\wedge {\left[x\right]}_{A_2}\subseteq X\wedge \dots \wedge {\left[x\right]}_{A_m}\subseteq X\},$$

(7)

$$\overline{\sum _{\begin{array}{c}i=1\end{array}}^m{A}_i^P}\left(x\right)={\left(\underline{\sum _{\begin{array}{c}i=1\end{array}}^m{A}_i^P}\left(x^c\right)\right)}^c$$

(8)

If $\underline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^P}\left(x\right)\ne \overline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^P}\left(x\right)$, then $\underline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^P}\left(x\right)$ and $\overline{{\sum }_{\begin{array}{c}i=1\end{array}}^m{A}_i^P}\left(x\right)$ are called pessimistic multi-granularity rough sets.

2.3. Multi-Granularity Neighborhood Rough Sets

In the literature [25], Lin et al. proposed two types of neighborhood multi-granularity rough sets, which can be applied to deal with incomplete systems containing numerical and categorical attributes [29]. To simplify the problem, when dealing with incomplete systems, only the application of neighborhood multi-granularity rough sets to numerical data is considered.

Given $DS=(U,AT)$, where $AT=\{A_k\mid k\in \{1,2,\dots ,m\}\}$, $U=\{x_i\mid i\in \{1,2,\dots ,n\}\}$, $X\subseteq U$, in the optimistic neighborhood multi-granularity rough sets, the neighborhood multi-granularity approximation of X is defined as:

$$\underline{\sum _{\begin{array}{c}i=1\end{array}}^m{N}_i^O}\left(x\right)=\{x_i\in U\mid \delta A_1\left(x_i\right)\subseteq X\vee \delta A_2\left(x_i\right)\subseteq X\vee \dots \vee \delta A_k\left(x_i\right)\subseteq X\}$$

(9)

$$\overline{\sum _{\begin{array}{c}i=1\end{array}}^m{N}_i^O}\left(x\right)=\{x_i\in U\mid \delta A_1\left(x_i\right)\cap X\ne \varnothing \wedge \delta A_2\left(x_i\right)\cap X\ne \varnothing \wedge \dots \wedge \delta A_m\left(x_i\right)\cap X\ne \varnothing \}$$

(10)

where $\delta A_k\left(x_i\right)$ is the neighborhood granularity of $x_i$, based on the granularity structure $A_k$.

Given $DS=(U,AT)$, where $AT=\{A_k\mid kin\{1,2,\dots ,m\}\}$, $U=\{x_i\mid iin\{1,2,\dots ,n\}\}$, $X\subseteq U$, in the pessimistic neighborhood multi-granularity rough sets, the neighborhood multi-granularity approximation of X is defined as:

$$\underline{\sum _{\begin{array}{c}i=1\end{array}}^m{N}_i^P}\left(x\right)=\{x_i\in U\mid \delta A_1\left(x_i\right)\subseteq X\wedge \delta A_2\left(x_i\right)\subseteq X\wedge \dots \wedge \delta A_k\left(x_i\right)\subseteq X\}$$

(11)

$$\overline{\sum _{\begin{array}{c}i=1\end{array}}^m{N}_i^P}\left(x\right)=\{x_{i}inU\mid \delta A_1\left(x_i\right)\cap X\ne \varnothing \vee \delta A_2\left(x_i\right)\cap X\ne \varnothing \vee \dots \vee \delta A_k\left(x_i\right)\cap X\ne \varnothing \}$$

(12)

where $\delta A_k\left(x_i\right)$ is the neighborhood granularity of $x_i$, based on the granularity structure $A_k$.

2.4. Supervised Attribute Reduction

It is well known that neighborhood rough sets are often used in supervised learning tasks, especially in enhancing generalization performance and reducing classifier complexity.[47] The advantage of attribute reduction lies in its easy adaptation to different practical application requirements, hence a variety of forms of attribute reduction have emerged in recent years. For neighborhood rough sets, information gain and split information value are two metrics that can be used to further explore the forms of attribute reduction.

Given the data $DS=\langle U,AT,d,\delta \rangle$, for any $A\subseteq AT$, the neighborhood information gain of D based on A is defined as:

$$I{G}_{NRS}(d,A)=H_{NRS}-H_{NRS}(d,A)$$

(13)

Here, $H_{NRS}$ is the entropy of the entire dataset D, calculated based on the distribution under neighborhood lower or upper approximation. $H_{NRS}(d,A)$ is the expected value of uncertainty considering attribute A, defined as:

$$H_{NRS}(d,A)=-\frac{1}{\left|U\right|}\sum _{x\in U}|{\delta }_s\left(x\right)\cap {\left[x\right]}_d|log\frac{|{\delta }_s\left(x\right)\cap {\left[x\right]}_d|}{|{\delta }_s\left(x\right)|}$$

(14)

Given the data $DS=\langle U,AT,d,\delta \rangle$, for any $A\subseteq AT$, the neighborhood split information value of d based on A is defined as:

$$S{I}_{NRS}(d,A)=-\sum _{j=1}^n\frac{|{\delta }_A\left(X_j\right)|}{\left|U\right|}{log}_2\frac{|{\delta }_A\left(X_j\right)|}{\left|U\right|}$$

(15)

Here ${\delta }_A\left(X_j\right)$ represents the sample set $X_j$ within the neighborhood formed by attribute A, and n is the number of different neighborhoods formed by A.

The combination of neighborhood information gain and split information value helps to more comprehensively assess the impact of attributes on dataset classification, thereby making more effective decisions in attribute reduction.

2.5. Unsupervised Attribute Reduction

It’s widely recognized that supervised attribute reduction necessitates the use of sample labels, which are time-consuming and expensive to obtain in many practical tasks. [48] In contrast, unsupervised attribute reduction does not require these labels, hence it has received more attention recently.

In unsupervised attribute reduction, if it is necessary to measure the importance of attributes, one can construct models by introducing pseudo-label strategies and using information gain and split information as metrics.

Given unsupervised data $IS=\langle U,AT\rangle$ and $\delta$, for any $A\subseteq AT$, the unsupervised information gain based on A is defined as:

$$I{G}_{NRS}(d,A)=H_{NRS}-H_{NRS}(d,A)$$

(16)

where $H_{NRS}(d,A)$ is the expected value of uncertainty considering attribute A, defined as:

$$H_{NRS}(d,A)=\frac{1}{\left|A\right|}\sum _{a\in A}\left(H_{NRS}(d,A)\left(d^a\right)\right)$$

(17)

$d_a$ denotes the pseudo-label decision for samples generated using conditional attribute a.

Given unsupervised data $IS=\langle U,AT\rangle$ and $\delta$, for any $A\subseteq AT$, the unsupervised split information based on A is defined as:

$$S{I}_{NRS}(d,A)=\frac{1}{\left|A\right|}\sum _{a\in A}\left(S{I}_{NRS}(d,A)\left(d^a\right)\right)$$

(18)

Here, $d_a$ is a pseudo-label decision, recorded by using conditional attribute a for sample pseudo-labels.

These definitions provide a new method for evaluating attribute importance in an unsupervised setting. Information gain reflects the contribution of an attribute to data classification, while split information measures the degree of confusion introduced by an attribute in the division of the dataset. This approach helps in more effective attribute selection and reduction in unsupervised learning.

3. Proposed Method

3.1. Definition of Multi-Granularity Neighborhood Information Gain Ratio

Considering a dataset $DS=\langle U,AT,d,\delta \rangle$, with U representing the sample set, $AT$ indicating the attribute set, d denoting the decision attribute, and $\delta$ specifying the neighborhood radius.

For any $A\subseteq AT$, the multi-granularity neighborhood information gain ratio is defined as:

$${ϵ}_A\left(d\right)=\frac{S{I}_{NRS}(d,A)}{e^{\left|I{G}_{NRS}(d,A)\right|}}\times W_A$$

(19)

where $S{I}_{NRS}(d,A)$ is the neighborhood split information quantity based on A, $e^{\left|I{G}_{NRS}(d,A)\right|}$ is the information gain for decision attribute d based on attribute A with the base of the natural logarithm, and $W_A$ is the granularity space coefficient of attribute A in the multi-granularity structure, reflecting its importance in the multi-granularity structure.

For the calculation of the granularity space coefficient, given a set of granularities $G_1,G_2,...,G_n$, the performance of attribute A under each granularity can be measured by the quantitative indicator $P_{G_i}\left(A\right)$. The definition of the granularity space coefficient $W_A$ is as follows:

$$W_A=\frac{{\sum }_{i=1}^n{\beta }_i·P_{G_i}\left(A\right)}{{\sum }_{i=1}^n{\beta }_i}$$

(20)

where ${\beta }_i$ is the granularity space allocated to each granularity $G_i$, reflecting the importance of different granularities. These granularity spaces are usually determined based on the specific background knowledge or experimental verification of the problem.

The granularities $G_1,G_2,...,G_n$ in the multi-granularity structure are determined according to the data characteristics, problem requirements, etc., and each granularity reflects different levels or details of the data. When calculating the granularity space coefficient, the performance of the attribute under different granularities is considered, in order to more accurately reflect its importance in the multi-granularity structure.

Neighborhood rough set is a method for dealing with uncertain and fuzzy data, which uses neighborhood relations instead of the indiscernible relations in traditional rough sets. In this method, data is decomposed into different granularities, each representing different levels or details of the data. Information gain ratio is a method for measuring the importance of attributes in data classification. It is based on the concept of information entropy and evaluates the classification capability of an attribute by comparing the entropy change of the dataset with and without the attribute.

Therefore, $ϵ$ combines these two concepts, i.e., neighborhood information gain at different granularities and the split information value of attributes, to evaluate the importance of attributes in multi-granularity data analysis. This method not only considers the information gain of attributes but also their performance at different granularities, thus providing a more comprehensive method of attribute evaluation.

Given a decision system $DS$ and a threshold $\theta \in [0,1]$, an attribute A is considered significant if it satisfies the following conditions:

• $\frac{{ϵ}_A\left(d\right)}{{ϵ}_{AT}\left(d\right)}\ge \theta$;
• There is no proper subset $A^{\prime }$ of attribute A such that $\frac{{ϵ}_A^{\prime }\left(d\right)}{{ϵ}_{AT}\left(d\right)}<\theta$.

In this definition, significant attributes are determined based on their contribution to the information gain ratio, aiming to select attributes that are informative yet not redundant for the decision-making process. This method is based on greedy search techniques for attribute reduction and helps identify attributes that significantly impact the decision outcome.

Given a dataset $DS=\langle U,AT,d\rangle$, where U is the set of objects, $AT$ is the set of conditional attributes, and d is the decision attribute. For any attribute subset $A\subseteq AT$ and any $a\in AT-A$ (i.e., any attribute not in A), the significance of attribute a regarding the multi-granularity neighborhood information gain ratio is defined as follows:

$$Si{g}_{{ϵ}_a\left(d\right)}={ϵ}_{A\cup \left\{a\right\}}\left(d\right)-{ϵ}_A\left(d\right)$$

(21)

The aforementioned significance function suggests that an increase in value enhances the importance of a conditional attribute, making it more probable to be included in the reduction set. For example, if $Si{g}_{{ϵ}_{a}1\left(d\right)}<Si{g}_{{ϵ}_{a}2\left(d\right)}$, where $a1,a2\in AT-A$, then ${ϵ}_{A\cup \left\{a1\right\}}\left(d\right)<{ϵ}_{A\cup \left\{a2\right\}}\left(d\right)$. Such a result indicates that choosing $a2$ to join A would lead to a higher multi-granularity neighborhood information gain ratio compared to $a1$.

3.2. Detailed Algorithm

Based on the significance function, Algorithm is designed to find the $ϵ$-reduct.

To streamline the analysis of the computational complexity for Algorithm , we initiate by applying k-means clustering to generate pseudo labels for the samples. With T denoting the iteration count for k-means clustering and k indicating the cluster count, the complexity of creating pseudo labels is $O(k·T·|U|·|AT\left|\right)$, where $\left|U\right|$ is the total number of samples, and $\left|AT\right|$ signifies the attribute count. Subsequently, the calculation of ${ϵ}_{A\cup a}\left(d\right)$ occurs no more than $(1+|AT\left|\right)·\left|AT\right|/2$ times. In conclusion, the computational complexity of Algorithm 1 equates to $O\left(\frac{{\left|U\right|}^2·{\left|AT\right|}^3}{2}+k·T·\left|U\right|·\left|AT\right|\right)$.

4. Experimental Analysis

4.1. Dataset Description

To evaluate the performance of the proposed measure, 15 UCI datasets are used in this experiment. Table 1 summarizes the statistical information of these datasets.

4.2. Experimental Configuration

The experiment is performed on a personal computer running Windows 11, featuring an Intel Core i5-12500H processor (2.50 GHz) with 16.00 GB RAM. MATLAB R2023a served as the development environment.

In this experiment, a double means algorithm is adopted to recursively allocate attribute granularity space, while utilizing the k-means clustering method [38] to generate pseudo labels for samples, and the information gain ratio as the criterion for evaluating attribute reduction. Notably, the selected k-value needs to match the number of decision categories in the dataset. Moreover, the effect of the neighborhood rough set is significantly influenced by the preset radius size. To demonstrate the effectiveness and applicability of the proposed method, a series of experiments are designed using 20 different radius values, incremented by 0.02, ranging from 0.02 to 0.40. Through 10-fold cross-validation, the simplified reasoning process is validated. Specifically, for each specific radius, the dataset was divided into ten subsets, nine for training and one for testing. This cross-validation process was repeated 10 times to ensure that each subset had the opportunity to serve as the test set, thereby evaluating the classification performance and ensuring the reliability and stability of the model.

In the experiment, the proposed measure is compared with six advanced attribute reduction algorithms using Regression Trees (CART) [36], K-Nearest Neighbors (KNN, K=3) [41], and Support Vector Machines (SVM) [37]. The performance of the reducer is evaluated in aspects of the stability, accuracy, and timeliness of classification, as well as the stability of reduction. The attribute reduction algorithms included for comparison are:

MapReduce-Based Attribute Reduction Algorithm (MARA) [40]

Robust Attribute Reduction Based On Rough Sets (RARR) [42].

Bipolar Fuzzy Relation System Attribute Reduction Algorithms (BFRS) [43]

Attribute Group (AG) [44];

Separability-Based Evaluation Function (SEF) [45];

Genetic Algorithm-based Attribute Reduction (GAAR) [46].

4.3. Comparison of Classification Accuracy

In this part, the classification accuracy of each algorithm is evaluated using KNN, SVM, and CART for predicting test samples. Regarding attribute reduction algorithms, within a decision system $DS$, the definition of classification accuracy post-reduction is as follows:

$$Ac{c}_{red}=\frac{\left|\right\{x_i\in red|{Pre}_{red}\left(x_i\right)=d\left(x_i\right)\}|}{\left|U\right|},$$

(22)

where, ${\mathrm{Pre}}_{red}\left(x_i\right)$ is the predicted label for $x_i$ using the reduced set $red$.

Table 2 and Figure 1 present the specific classification accuracy outcomes for each algorithm across 15 datasets. From these observations, several insights can be readily inferred:

• For most datasets, the classification accuracy associated with NRS-$ϵ$ is superior to other comparison algorithms, regardless of whether the KNN, SVM, or CART classifier is used. For example, in the "Car Evaluation (ID: 6)" dataset, when using the CART classifier, the classification accuracies of NRS-$ϵ$, MARA, RARR, BFRS, AG, SEF, and GAAR are 0.5039, 0.4529, 0.4157, 0.494, 0.4886, 0.4909, 0.4719 respectively; when using the KNN classifier, the classification accuracies of NRS-$ϵ$, MARA, RARR, BFRS, AG, SEF, and GAAR are 0.6977, 0.6584, 0.535, 0.6747, 0.6675, 0.6586, 0.6579 respectively; when using SVM, the classification accuracies of NRS-$ϵ$, MARA, RARR, BFRS, AG, SEF, and GAAR are 0.5455, 0.4307, 0.368, 0.4737, 0.4698, 0.4718, 0.4923 respectively. Therefore, NRS-$ϵ$ derived simplifications can provide effective classification performance.
• Examining the average classification accuracy per algorithm reveals that the accuracy associated with NRS-$ϵ$ is on par with, if not exceeds, that of MARA, RARR, BFRS, AG, SEF, and GAAR. When using the CART classifier, the average classification accuracy of NRS-$ϵ$ is 0.8012, up to 29.28% higher than other algorithms; when using the KNN classifier, the average classification accuracy of NRS-$ϵ$ is 0.8169, up to 34.48% higher than other algorithms; when using SVM, the average classification accuracy of NRS-$ϵ$ is 0.80116, up to 36.38% higher than other algorithms.

4.4. Comparison of Classification Stability

Similar to the evaluation of classification accuracy, this section explores the classification stability obtained by analyzing the classification results of seven different algorithms, including experiments with CART, KNN, and SVM classifiers. In a decision system $DS=\langle U,AT,d,\delta \rangle$, assume the set U is equally divided into z mutually exclusive groups of the same size (using 10-fold cross-validation, so $z=10$), that is $U_1,\dots ,U_{\tau },\dots ,U_z$ ($1\le \tau \le z$). Then, the classification stability based on redundancy reduction ${\mathrm{red}}_{\tau }$ (obtained by removing $U_{\tau }$ from the set U) can be represented as:

$${Stab}_{class}=\frac{2}{z·(z-1)}\sum _{\tau =1}^{z-1}\sum _{{\tau }^{\prime }=\tau +1}^{z}Exa({red}_{\tau },{red}_{{\tau }^{\prime }})$$

(23)

where $\mathrm{Exa}({red}_{\tau },{red}_{{\tau }^{\prime }})$ measures the consistency between two classification results, which can be defined according to Table 3.

In Table 3, ${Prered}_{\tau }\left(x\right)$ represents the predicted label of x obtained by ${\mathrm{red}}_{\tau }$. The symbols ${\psi }_1$, ${\psi }_2$, ${\psi }_3$, and ${\psi }_4$ respectively represent the number of samples that satisfy the corresponding conditions in Table Table 4. Based on this, $Exa({red}_{\tau },{red}_{{\tau }^{\prime }})$ is defined as

$$Exa({red}_{\tau },{red}_{{\tau }^{\prime }})=\frac{{\psi }_1+{\psi }_4}{{\psi }_1+{\psi }_2+{\psi }_3+{\psi }_4}.$$

(24)

The classification stability index reflects the degree of deviation of prediction labels when data perturbation occurs. Higher values of classification stability mean more stable prediction labels, indicating better quality of the corresponding reduction. Improvements in classification stability mean increased stability of prediction label results and reduced interference with training samples. After analyzing the 15 datasets using these three classifiers, Table Table 4 and Figure 2 present the findings of each algorithm in terms of classification stability. It should be noted that the classification stability index reflects the degree of change in prediction labels when data is perturbed. Higher classification stability values indicate more stable prediction labels, meaning the related redundancy reduction has a higher quality.

• Across many datasets, the NRS-$ϵ$ algorithm exhibits leading performance compared to other algorithms in terms of classification stability, playing a leading role. For example, in the "Iris Plants Database (ID: 2)" dataset, significant differences in classification accuracy were observed under different classifiers for NRS-$ϵ$ and other algorithms: when using the CART classifier, the accuracy of NRS-$ϵ$ reached 0.7364, while MARA, RARR, BFRS, AG, SEF, and GAAR had accuracies of 0.6794, 0.7122, 0.6981, 0.7244, 0.7130, and 0.7276 respectively; when using the KNN classifier, the accuracy of NRS-$ϵ$ was 0.8357, with MARA, RARR, BFRS, AG, SEF, and GAAR algorithms having accuracies of 0.6380, 0.8349, 0.8155, 0.8145, 0.8253, and 0.8246 respectively; when using the SVM classifier, the accuracies of NRS-$ϵ$, MARA, RARR, BFRS, AG, SEF, and GAAR were 0.8918, 0.6581, 0.8774, 0.8771, 0.8748, 0.8852, 0.8783 respectively.
• Regarding average classification accuracy, the stability of NRS-$ϵ$ markedly surpasses that of competing algorithms. Specifically, when using the CART classifier, the classification stability of NRS-$ϵ$ was 0.8228, up to 12.51% higher than other methods; when using the KNN classifier, its classification stability was 0.8972, up to 25.14% higher than other methods; and through the SVM classifier, the classification stability of NRS-$ϵ$ was 0.9295, up to 14.61% higher than other methods.

4.5. Comparisons of Elapsed Time

In this section, the time required for attribute reduction by different algorithms is compared. The results are shown in the Table 5.

An increase in the value of dimensionality reduction stability correlates with an extended length of reduction. Through an in-depth analysis of the table below, the findings listed below can be easily derived. The reduction length of NRS-$ϵ$ is longer, suggesting that there’s a need to enhance the algorithm’s time efficiency throughout the simplification process.

When analyzing the average processing time performance of algorithms, from the perspective of average time consumed, it is noteworthy that the value of NRS-$ϵ$ is reduced by 97.23% and 48.86% compared to RARR and GAAR, respectively. Taking the dataset "Car Evaluation (ID: 6)" as an example, the time consumed by NRS-$ϵ$, MARA, RARR, BFRS, AG, SEF, and GAAR are 122.1212 seconds, 6.9838 seconds, 421.1056 seconds, 154.8219 seconds, 31.4599 seconds, 33.3661 seconds, and 54.0532 seconds, respectively. Hence, under certain conditions, the time NRS-$ϵ$ takes for attribute reduction is less compared to RARR and BFRS.

Based on the discussion, it is evident that while our novel algorithm exhibits better time efficiency compared to RARR and BFRS on certain datasets, the speed of NRS-$ϵ$ requires further enhancement.

4.6. Comparison of Attribute Dimensionality Reduction Stability

In this section, the attribute dimensionality reduction stability related to 15 datasets is presented. Table 6 shows that the dimensionality reduction stability of NRS-$ϵ$ is slightly lower than GAAR and SEF but still maintains a leading position. Compared to MARA, RARR, BFRS, and AG, the average dimensionality reduction stability value of NRS-$ϵ$ has increased by 100.2%, 49.89%, 27.19%, and 14.15%, respectively, while it only decreased by 19.323% and 6.677% compared to GAAR and SEF.

Although NRS-$ϵ$ does not fall short of GAAR and SEF’s results in terms of dimensionality reduction stability on many datasets, in some cases, its results in attribute dimensionality reduction are superior to the six advanced algorithms. For example, for the "Letter Recognition (ID: 15)" dataset, the dimensionality reduction stability of NRS-$ϵ$, MARA, RARR, BFRS, AG, SEF, and GAAR were 0.8608, 0.6001, 0.4011, 0.7882, 0.6549, 0.7723, and 0.7442, respectively. Compared to other algorithms, the results of NRS-$ϵ$ improved by 43.47%, 115.2%, 9.211%, 31.44%, 11.46%, and 15.67%, respectively. Thus, it’s important to recognize that employing NRS-$ϵ$ favors the selection of attributes better aligned with variations in samples.

5. Conclusion and Future Expectations

To address the challenges of high-dimensional data, a new attribute reduction search strategy is proposed in this paper. Unlike traditional methods, on the one hand, by integrating multi-granularity into a new model, the new model becomes more flexible and adaptable to data features at different levels. On the other hand, the new model combines supervised and unsupervised perspectives, overcoming the limitations of single-attribute measurement methods.

Through experiments on 15 UCI datasets, it is not difficult to find that our proposed strategy possesses excellent classification performance as well as strong stability during the dimensionality reduction process. However, our strategy still has some shortcomings, which are specifically manifested in the following aspects:

• Although the use of integrated measurement methods has improved the quality of attribute selection, it has also correspondingly increased the time cost of the selection process. For this reason, future work can explore introducing more acceleration technologies to optimize efficiency and reduce the required time.
• Given the high universality of the search framework proposed in this paper, future research can attempt to use different rough set-based fundamental measurements to replace the current information gain ratio, to assess the performance of different measurement methods in classification performance.