MFDRRS: Modified Fisher Discriminant Ratio Based Rough Set Heuristic Attribute Reduction for Nominal Data

1. Introduction

In the era of big data, the sheer volume of information is accompanied by substantial redundancy. Effectively extracting actionable knowledge from data remains a fundamental challenge in data mining, as the knowledge discovered by conventional data mining approaches often fails to directly support meaningful decision-making actions[1,2]. Knowledge reduction, particularly attribute reduction in rough set theory, has emerged as an important technique for eliminating redundant and irrelevant attributes while preserving the essential decision-making information of the original data, making it particularly valuable for high-dimensional data analysis and knowledge discovery[3]. In the context of machine learning, knowledge reduction is also referred to as feature selection or attribute reduction[4,5]. The core objective of attribute reduction is to minimize the size of the feature set as much as possible while maintaining the classification ability of the classifier unchanged. To achieve this goal, the researchers have proposed numerous processing methods, which can be broadly classified into the following categories: filter, wrapper and embedded methods[6]. Filter methods evaluate features based on their statistical properties and are independent of specific learning algorithms, offering high computational efficiency and scalability. Common examples include Fisher Score[7], Relief-based algorithms[8], mutual information[9], and minimum redundancy maximum relevance (mRMR)[10] etc. However, since these methods typically evaluate features individually or rely on local statistics, they fail to adequately capture inter-feature correlations and joint effects, limiting their performance on complex data. Wrapper methods evaluate feature subsets based on the predictive performance of a learning algorithm, employing search strategies such as sequential search, genetic algorithms, and particle swarm optimization to identify optimal combinations. Although they often achieve high classification accuracy, the need for iterative model training incurs substantial computational overhead, which severely limits their applicability to large-scale, high-dimensional data. Embedded methods integrate feature selection into model training by imposing regularization constraints (e.g., L1 and L2 norms), as exemplified by LASSO[11], Elastic Net[12], tree-based models[13], and sparse learning approaches[14]. By balancing high predictive performance with computational efficiency, these methods have become a prominent research focus in supervised feature selection. The growing prevalence of high-dimensional data and complex structures has made unsupervised feature selection a prominent research topic. Without class labels, these methods assess feature relevance by preserving the local manifold, global geometry, or clustering structure of the data. Typical techniques—Laplacian Score[15], SPEC[16], MCFS[17], NDFS[18], and UDFS[19]—select features by constructing similarity graphs or graph Laplacian matrices to retain manifold structure.

The proliferation of high-dimensional datasets across genomics, healthcare, and other data-intensive applications has introduced significant challenges for feature selection, including increased computational costs, substantial feature redundancy, and the difficulty of effectively capturing complex dependencies among features[20,21,22,23]. Requiring no prior knowledge or distributional assumptions, rough sets are inherently well-suited for high-dimensional complex data, providing effective feature reduction and strong interpretability [5]. In the framework of Pawlak’s rough set theory[24], attribute reduction is formalized through the dependency degree $\gamma (K,d)$, which measures the proportion of objects correctly classified by an attribute subset $K\subseteq C$, where C is the full set of conditional attributes and d is the decision attribute. A reduct is any subset $R\subseteq C$ that satisfies $\gamma (R,d)=\gamma (C,d)$ and no subset of R achieves the same dependency degree. However, the search for all reducts is NP-hard[25,26], motivating the use of heuristic strategies. The two most widely used heuristics are greedy forward selection [27] and greedy backward elimination [28]. Both methods rely on the monotonicity of $\gamma (K,d)$, iteratively adding or removing the attribute that best improves (or least reduces) the dependency degree. While simple and effective in practice, these methods suffer from two critical limitations that restrict their utility in real-world applications. (1) lack of principled criteria for attribute ranking. Without principled criteria for prioritizing attributes, both methods rely solely on marginal gains in $\gamma$, which can be misleading when attributes differ in scale, distribution, or cardinality. An attribute with only modest conditional gain may still exhibit strong statistical discriminability—a property that pure rough-set heuristics cannot capture. Most existing rough-set heuristics treat candidate attributes uniformly[29], lacking a statistical prior to distinguish informative features from noisy ones. (2) Ad-hoc stopping criteria lacking principled termination conditions. Both forward and backward searches terminate based on user-specified thresholds (e.g., minimum gain $\tau$). These hyperparameters have no principled foundation and require per-dataset tuning, placing a considerable burden on practitioners. As Liu et al. [30] observe in their evaluation of rough-set heuristics, "the sensitivity of final reducts to the stopping threshold necessitates extensive parameter tuning across diverse datasets, undermining reproducibility". Although mutual information-based methods[31,32] address the first limitation by leveraging information-theoretic quantities, they do not directly quantify rough-set classification capability and capture dependence without considering class-separability statistics—factors that are particularly important, where interactions between attribute values and class labels demand careful quantification.

Motivated by these limitations, we propose MFDRRS—a new method that integrates Fisher discriminant ratio (FDR) with rough sets to solve attribute reduction problems. Unlike heuristic reductions based on metrics such as frequency and entropy, we introduce the Fisher discriminant ratio to guide candidate attribute generation. The Fisher discriminant ratio provides a transparent and interpretable metric that quantifies how well an attribute separates classes relative to between-class and within-class variability. A higher FDR score indicates stronger class-separability, implying that the attribute is more representative than others. To our knowledge, no prior work has systematically incorporated FDR—the cornerstone of Fisher’s linear discriminant analysis—into the rough-set heuristic framework. Although Zhou et al.[33] introduced hypothesis testing in rough sets for small-sample settings, and Chen et al.[34] integrated rough sets with SVM weighting, neither approach utilized FDR as a ranking criterion or pre-filtering mechanism. Our approach fills this gap by treating FDR as a statistical surrogate for rough-set discriminability. Furthermore, we replace the fixed minimum-gain parameter $\tau$ with a dynamic termination criterion based on the ranked FDR distribution, since candidate attributes in the lower tail yield diminishing returns. This data-driven criterion adjusts to the inherent difficulty of each dataset, thereby mitigating the reproducibility issues raised by Liu et al.[30]. Lastly, we reduce the computational cost of dependency-degree evaluations by pruning attributes with negligible between-class scatter relative to within-class scatter prior to greedy selection. This is particularly beneficial for datasets with many nominal attributes, where evaluating $\gamma (K,d)$ becomes computationally expensive.

Our main contributions are summarized as follows:

• We propose the first integration of the Fisher discriminant ratio into rough-set attribute reduction, providing a principled statistical criterion for attribute ranking that is absent in existing heuristic methods.
• We introduce an FDR-driven stopping rule that substitutes the ad-hoc minimum-gain threshold with a data-driven criterion based on FDR rankings, thereby improving robustness and cross-dataset generalizability.
• We further assessed the proposed algorithm through experiments to confirm its effectiveness and efficiency. Experimental results demonstrate that our algorithm achieves more compact reductions with fewer attributes, maintains competitive dependency degrees, and avoids the over-selection characteristic of classical backward elimination.

The remainder of this paper is organized as follows. Section 2 reviews related work on rough-set attribute reduction and statistical feature ranking. Section 3 introduces the necessary rough-set concepts and extends Fisher Discriminant Ratio to nominal attributes. Section 4 presents the MFDRRS algorithm and its complexity analysis. Section 5 details the experimental setup and reports comparative results on UCI and synthetic datasets. Finally, Section 6 draws conclusions and outlines future research directions.

2. Related Work

2.1. Rough Set-Based Attribute Reduction

Pawlak’s original formulation[24] defines attribute reduction via indiscernibility relations and positive region approximation. However, the strictness of the equivalence relation makes attribute reduction vulnerable to outliers and ill-suited for practical applications[4,5]. To address this, numerous studies have relaxed the equivalence constraints and redefined the reduct, including probabilistic rough set model[35,36], variable precision rough set model[37,38], covering rough set model[39,40], fuzzy rough set model[41,42,43], and algebra viewpoint attribute reduction[44], etc. As computing all reducts is NP-hard, reduction methods can be divided, from a computability viewpoint, into two categories: data structure optimization and heuristic search. The first category primarily employs data structure optimization in reduction computation, e.g., discernibility matrix[45,46,47], discernibility hash[48], binary tree structure[49], and tree structure[50,51] etc. The second approach relies on heuristic information from candidate attributes to expedite the reduct process. For example, Qian et al. [52] proposed the Pangu Sorting algorithm, a greedy forward selection strategy that iteratively adds the attribute with the maximum marginal gain in dependency degree. Li et al.[53] introduced entropy-based measures for continuous attributes. Similar approaches include information-theoretic methods that employ heuristic measures [54,55,56,57]. Recent efforts have attempted to enhance heuristic stability. Zhao et al. [26] studied optimal reduct approximations via granular computing and showed that basic greedy heuristics yield unstable selections under minor data perturbations. Yao and Wang [27] surveyed uncertainty measures for rough-set feature selection, observing that dependency degree is the most commonly used criterion, despite its poor sensitivity to attribute interactions. Feng et al. [58] recently explored ensemble methods for reduct aggregation to address instability, albeit with higher computational overhead. For large-scale data, Wang et al.[59] proposed approximate rough-set heuristics with sampling to reduce dependency-degree cost, sacrificing accuracy for scalability.

2.2. Statistical Feature Selection

Beyond rough-set frameworks, statistical methods have long informed feature selection. Benjamini and Hochberg [60] introduced the FDR-controlling step-up procedure, which has become a standard tool in genomics and high-dimensional data analysis. Subsequent work extended FDR control to adaptive procedures [61] and independent filtering [62]. Zhu et al. [31] performed a systematic comparison of mutual information estimators for categorical features in the machine learning community, revealing that MI-based rankings are sensitive to discretization choices and sample size. They advised using statistical independence tests alongside MI to prevent false selections. Wang et al. [32] similarly extended joint mutual information approaches to better address redundancy, yet their method still suffers from the limitation that MI scores are not comparable across datasets with differing numbers of variables. Chen and Yu [63] introduced a statistical testing framework for nominal feature selection that uses chi-square tests for attribute screening prior to downstream modeling. Their approach aligns with our objective of integrating statistical significance into feature selection; however, it relies solely on p-values rather than effect-size measures such as FDR, which offer finer differentiation between moderately and strongly informative attributes.

Several studies have combined rough sets with statistical tests. Zhou et al.[33] used Fisher’s exact test within a rough-set framework for small-sample genomic data, applying multiple-testing correction to filter candidate attributes before greedy selection. While this represents an important step toward statistical rigor, their focus on exact tests for binary outcomes limits applicability to multi-class nominal settings. Moreover, they did not integrate FDR-like scatter metrics that capture both between-class separation and within-class compactness simultaneously. Chen et al.[34] integrated support vector machines with rough sets for feature weighting, assigning weights based on SVM coefficients followed by rough-set reduct extraction. However, this wrapper-style approach incurs additional computational overhead and does not provide a statistical guarantee on individual attribute importance. Furthermore, it assumes numeric encoding even for nominal inputs, potentially introducing artificial ordinal relationships.

3. Preliminaries

This section lays the groundwork by introducing the essential concepts of rough sets, which will inform the research problem formulation and discussion in the following sections.

Definition 1

(Decision Information System [24]). A decision information system is defined as a quadruple $S=(U,A,V,f)$, where:

(1). 

$U={\left\{{\mathit{x}}_i\right\}}_{i=1}^n$ is a non-empty finite set which represents the object domain, where each ${\mathit{x}}_i$ is an object and n is the total number of objects.

(2). 

$A={\left\{a_i\right\}}_{i=1}^m$ is a non-empty finite set which represents the attribute domain, where each $a_i$ is an attribute and m is the total number of attributes. The set A is devided into conditonal attributes C and decision attributes D, where $A=C\cup D$ and $C\cap D=\varnothing$.

(3). 

$V={\bigcup }_{a\in A}{V}_a$ is the domain of attribute values, where $V_a$ is the set of values for attribute a.

(4). 

$f:U\times A\to V$ is an information function which maps each object-attribute pair to a value in the value domain V.

Table 1 presents a decision information system for influenza diagnosis, where $U=\{x_1,x_2,x_3,x_4,x_5,x_6\}$ is the object domain, $C=\{\mathrm{Fever},\mathrm{Cough},\mathrm{Headache}\}$ is the conditional attribute domain, $D=\left\{\mathrm{Diagnosis}\right\}$ is the decision attribute domain. According to this diagnosis system, we can obtain some calculation results, such as: $V_{Fevel}=\{High,Normal\}$, $V_{Cough}=\{Yes,No\}$, $V_{Headache}=\{Yes,No\}$, $V_{Diagnosis}=\{Flu,Healthy\}$, $f({\mathit{x}}_1,Fevel)=High$ and $f({\mathit{x}}_5,Cough)=No$ etc.

Definition 2

(Indiscernibility Relation [24]). Given a subset of attributes $B\subseteq C$, the indiscernibility relation $IND\left(B\right)$ induced by B is defined as:

$$IND\left(B\right)=\{({\mathit{x}}_i,{\mathit{x}}_j)\in U\times U|f({\mathit{x}}_i,a)=f({\mathit{x}}_j,a),\forall a\in B\}.$$

(1)

The relation $IND\left(B\right)$ is an equivalence relation that partitions the univers U into equivalence classes,

$$U/IND\left(B\right)=\left\{{\left[{\mathit{x}}_i\right]}_B\right|{\mathit{x}}_i\in U\},$$

(2)

where

$${\left[{\mathit{x}}_i\right]}_B=\{{\mathit{x}}_j\in U|({\mathit{x}}_i,{\mathit{x}}_j)\in IND\left(B\right)\}.$$

(3)

Consider the subset of attributes $B=\{\mathrm{Fever},\mathrm{Cough}\}\subseteq C$. Then, for example, the object pair $({\mathit{x}}_1,{\mathit{x}}_2)$ satisfies the indiscernibility relation $IND\left(B\right)$, and $U/IND\left(B\right)=\{({\mathit{x}}_1,{\mathit{x}}_2),\left({\mathit{x}}_3\right),\left({\mathit{x}}_5\right),({\mathit{x}}_4,{\mathit{x}}_6)\}$.

Definition 3

(Positive Region [24]). Given a subset of attributes $B\subseteq C$ and a decision attribute $d\in D$, the positive region $PO{S}_B\left(d\right)$ is defined as:

$$PO{S}_B\left(d\right)=\bigcup _{{\left[{\mathit{x}}_i\right]}_B\in U/IND\left(B\right)}\left\{{\left[{\mathit{x}}_i\right]}_B\right|{\left[{\mathit{x}}_i\right]}_B\subseteq {\left[{\mathit{x}}_j\right]}_d,\exists {\left[{\mathit{x}}_j\right]}_d\in U/IND\left(d\right)\}.$$

(4)

Definition 4

(Dependency Degree [24]). Given a subset of attributes $B\subseteq C$ and a decision attribute $d\in D$, the dependency degree ${\gamma }_B\left(d\right)$ is defined as:

$${\gamma }_B\left(d\right)={\displaystyle \frac{|PO{S}_B\left(d\right)|}{\left|U\right|}}\in [0,1].$$

(5)

Definition 5

(Reduct [24]). Given a subset of attributes $B\subseteq C$ and a decision attribute $d\in D$, the subset B is called a reduct of A with respect to d if it satisfies the following two conditions:

(1). 

${\gamma }_B\left(d\right)={\gamma }_A\left(d\right)$, where ${\gamma }_B\left(d\right)$ is the dependency degree of B with respect to d.

(2). 

$\forall B^{\prime }\subset B$, ${\gamma }_{B^{\prime }}\left(d\right)<{\gamma }_A\left(d\right)$.

4. Proposed Method

The proposed FDRRS method integrates FDR into rough-set-based attribute reduction through a three-stage process: statistical discriminability assessment of each condition attribute, significance-based pre-screening of the attribute space, and dependency-driven greedy forward selection with a data-adaptive stopping criterion. We detail each stage below.

4.1. Modified Fisher Discriminant Ratio

To quantify the discriminative power of each condition attribute, we adapt the Fisher discriminant ratio—a classical linear discriminant analysis metric—to nominal data. The traditional Fisher discriminant ratio [7,64] is designed for numerical features, as its computation relies on class means and variances. Since nominal attributes have neither numerical ordering nor meaningful statistical moments such as mean and variance, the conventional FDR cannot be directly applied to nominal data.

Definition 6

(Indicator function). Given a nominal attribute $a_j\in A$ with value domain $V_{a_j}=\{v_{j1},\dots ,v_{j{r}_j}\}$ of cardinality $r_j$, the indicator function for each value $v_{jk}\in V_{a_j}$ is defined as:

$${\mathbb{I}}_{jk}\left(x\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{a}_j\left(x\right)=v_{jk},\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(6)

Definition 7

(Conditional frequencies and marginal frequencies). Given a nomial attriubte $a_j\in A$ with value domain $V_{a_j}=\{v_{j1},\dots ,v_{j{r}_j}\}$ of cardinality $r_j$, the conditional frequencies and marginal frequencies of attribute value $v_{jk}$ are defined as:

$${\mu }_{jk}^{\left(c\right)}={\displaystyle \frac{1}{|U_c|}}\sum _{x\in U_c}{\mathbb{I}}_{jk}\left(x\right),$$

(7)

$${\mu }_{jk}={\displaystyle \frac{1}{n}}\sum _{x\in U}{\mathbb{I}}_{jk}\left(x\right).$$

(8)

where $U_c=\{x\in U|f(D,x)=c\}$ denote the set of objects belonging to decision class c with $|U_c|$ objects.

Taking the decision information system in Table 1, for example, we can calculate the conditional frequencies and marginal frequencies of the attribute $a_1=‘\mathrm{Fever}’$ as follows. First, we divide the object domain U into two decision classes according to the decision attribute $D=‘\mathrm{Diagnosis}’$, i.e., $U_{Flu}=\{x_1,x_2,x_3\}$ and $U_{Healthy}=\{x_4,x_5,x_6\}$ and get the value set of attribute $a_1$: $V_{a_1}=\{v_{11}=‘\mathrm{High}’,v_{12}=‘\mathrm{Normal}’\}$. Then, the conditional frequencies of the attribute value $v_{11}=‘\mathrm{High}’(j=1,k=1)$ are calculated as

$${\mu }_{11}^{\left(Flu\right)}={\displaystyle \frac{1}{|U_{Flu}|}}\sum _{x\in U_{Flu}}{\mathbb{I}}_{11}\left(x\right)={\displaystyle \frac{1}{3}}(1+1+0)={\displaystyle \frac{2}{3}},$$

(9)

$${\mu }_{11}^{\left(Healthy\right)}={\displaystyle \frac{1}{|U_{Healthy}|}}\sum _{x\in U_{Healthy}}{\mathbb{I}}_{11}\left(x\right)={\displaystyle \frac{1}{3}}(0+1+0)={\displaystyle \frac{1}{3}}.$$

(10)

And, the marginal frequency of the attribute value $v_{11}=‘\mathrm{High}’$ is calculated as

$${\mu }_{11}={\displaystyle \frac{1}{n}}\sum _{x\in U}{\mathbb{I}}_{11}\left(x\right)={\displaystyle \frac{1}{6}}(1+1+0+0+1+0)={\displaystyle \frac{3}{6}}={\displaystyle \frac{1}{2}}.$$

(11)

Analogous to the FDR for continuous data, we define the intra-class and inter-class divergences for nominal attributes as follows.

Definition 8

(Intra-class and Inter-class Divergences). Given a nominal attribute $a_j\in A$ with value domain $V_{a_j}=\{v_{j1},\dots ,v_{j{r}_j}\}$ of cardinality $r_j$, the intra-class divergence $S_W\left(a_j\right)$ and inter-class divergence $S_B\left(a_j\right)$ are defined as:

$$S_B^{\left(j\right)}=\sum _c\left|U_c\right|{\displaystyle \sum _{k=1}^{r_j}}{\left({\mu }_{jk}^{\left(c\right)}-{\mu }_{jk}\right)}^2,$$

(12)

$$S_W^{\left(j\right)}=\sum _c\sum _{x\in U_c}{\displaystyle \sum _{k=1}^{r_j}}{\left({\mathbb{I}}_{jk}\left(x\right)-{\mu }_{jk}^{\left(c\right)}\right)}^2.$$

(13)

Definition 9

(Modified Fisher Discriminant Ratio, MFDR). Given a decision information system $S=(U,A,V,f)$ and a nominal attribute $a_j\in A$ with value domain $V_{a_j}=\{v_{j1},\dots ,v_{j{r}_j}\}$ of cardinality $r_j$, the modified Fisher discriminant ratio (MFDR) for nominal attributes $a_j$ is defined as:

$$MFDR\left(a_j\right)={\displaystyle \frac{S_B^{\left(j\right)}}{S_W^{\left(j\right)}+ϵ}},$$

(14)

where ϵ is a small positive constant to avoid division by zero.

$MFDR\left(a_j\right)$ quantifies how well $a_j$ separates decision classes relative to its inter- and intra-class variance; larger values indicate stronger discriminative power. This formulation naturally generalizes the continuous FDR to nominal data via indicator-variable encoding and remains computable for any attribute of finite cardinality.

4.2. MFDR Filtering

Given the MFDR scores for all conditional attributes C with non-increasing order, i.e.,

$$MFDR\left(a_1^{\prime }\right)\ge MFDR\left(a_2^{\prime }\right)\ge \dots \ge MFDR\left(a_m^{\prime }\right),$$

(15)

we can obtain a candidate attribute set and iteratively select the highest-ranked attribute. According to this filtering rule, we can also remove attributes with low MFDR scores that are unlikely to contribute meaningfully to class separability. Specifically, We define a filtered candidate set $C_{\mathrm{sig}}\subseteq C$ by applying a threshold $\theta \ge 0$:

$$C_{\mathrm{sig}}=\{a_j\in C\mid \mathrm{MFDR}\left(a_j\right)\ge \theta \}.$$

(16)

The parameter $\theta$ controls the aggressiveness of pre-screening: $\theta =0$ retains all attributes, while $\theta >0$ eliminates attributes whose discriminative power falls below the threshold. This step reduces the search space for subsequent greedy selection, thereby improving computational efficiency, particularly when the number of candidate attributes m is large and many exhibit negligible MFDR values.

4.3. MFDR-Guided Rough-Set Attribute Reduction Algorithm

Roughly speaking, the problem of attribute reduction solving by Pawlak’s rough set theory can be formulated as follows:

$$\begin{array}{cc}\hfill \underset{R}{max}\underset{\left|R\right|}{min}& {\gamma }_R\left(D\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\gamma }_R\left(D\right)={\gamma }_C\left(D\right),\hfill \end{array}$$

(17)

where $R\subseteq C$ is a reduct of the conditional attribute set C. By introducing the MFDR filtering step, we can solve the problem iteratively by selecting the attribute $a_j\in C_{\mathrm{sig}}$ with the highest MFDR score that maximally increases the dependency degree ${\gamma }_{R\cup \left\{a_j\right\}}\left(D\right)$ at each iteration. The process continues until no further attributes can be added without violating the condition ${\gamma }_R\left(D\right)={\gamma }_C\left(D\right)$ or until the MFDR score of remaining candidates falls below a dynamic stopping threshold derived from the distribution of MFDR scores. In other words, we can obtain the solution of the original problem in Equation(17) by iteratively solving the following sub-problems:

$$\begin{array}{cc}\hfill a^{*}=& arg\underset{a\in C_{\mathrm{sig}}\setminus R}{max}{\gamma }_{R\cup \left\{a\right\}}\left(D\right)\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\gamma }_{R\cup \left\{a\right\}}\left(D\right)>{\gamma }_R\left(D\right).\hfill \end{array}$$

(18)

Specifically, starting from the empty set $R=⌀$, we iteratively select the attribute $a^{*}$ from $C_{\mathrm{sig}}$ that maximizes the marginal gain in rough-set dependency degree. To avoid exhaustive search while maintaining reduction compactness, we employ a dual stopping criterion. Let Q3 denote the third quartile of the MFDR values ${\left\{{\mathrm{MFDR}}_a\right\}}_{a\in C_{\mathrm{sig}}}$. At each iteration, the selection process terminates if either of the following conditions holds:

• MFDR-based termination: $\mathrm{MFDR}\left(a^{*}\right)<\mathrm{Q}3\times \alpha$, where $\alpha \in (0,1)$ is a decay factor (default $\alpha =0.5$). This criterion leverages the statistical ranking to detect when the best remaining candidate no longer provides discriminative power substantially above the dataset median.
• Minimum-gain termination: $\gamma (R\cup \left\{a^{*}\right\},d)-\gamma (R,d)<\tau$, where $\tau >0$ is a small tolerance (default $\tau ={10}^{-3}$). This ensures that attributes contributing negligible improvement to the dependency degree are not included.

If either condition is satisfied, the algorithm terminates and outputs the current reduct R. Otherwise, it adds $a^{*}$ to R and proceeds.The complete algorithm is summarized in Algorithm 1.

Algorithm 1 MFDRRS: MFDR-Guided Rough-Set Attribute Reduction

Require:  Decision information system $S=(U,C\cup \{d\},V,f)$, MFDR threshold $\theta \ge 0$, min gain $\tau >0$, decay factor $\alpha \in (0,1)$ Ensure:  Reduct $R\subseteq C$ 1: For each $a_j\in C$, compute $\mathrm{MFDR}\left(a_j\right)$ via Eq. (14) 2: Sort MFDRs in descending order; select $C_{\mathrm{sig}}=\{a_j\mid \mathrm{MFDR}\left(a_j\right)\ge \theta \}$ 3: if  $C_{\mathrm{sig}}=⌀$  then  return⌀ 4: end if 5: $R\leftarrow ⌀$, ${\gamma }_{\mathrm{prev}}\leftarrow 0$ 6: while  $C_{\mathrm{sig}}\setminus R\ne ⌀$  do 7:     $a^{*}\leftarrow arg{max}_{a\in C_{\mathrm{sig}}\setminus R}\gamma (R\cup \left\{a\right\},d)$                ▹ Eq. (18) 8:     $\mathrm{gain}\leftarrow \gamma (R\cup \left\{a^{*}\right\},d)-{\gamma }_{\mathrm{prev}}$ 9:     if $\mathrm{gain}<\tau$ then break 10:     else if $\mathrm{MFDR}\left(a^{*}\right)<\mathrm{Q}3\times \alpha$ then break 11:     else 12:         $R\leftarrow R\cup \left\{a^{*}\right\}$, ${\gamma }_{\mathrm{prev}}\leftarrow \gamma (R,d)$ 13:     end if 14: end while 15: return  R

4.4. Complexity Analysis

Let $n=\left|U\right|$, $m=\left|C\right|$, $r_{max}={max}_j{r}_j$, and $r_d$ be the number of decision classes. Computing MFDR for each attribute requires $O(r_j·n)$ operations per attribute using class-wise aggregation, yielding a total of $O(n·m·r_{max})$. For binary attributes ($r_j=2$), this simplifies to $O(n·m)$. Sorting MFDR values requires $O(mlogm)$ time. Computing the third quartile is $O\left(m\right)$ using a linear-time selection algorithm. Each iteration computes $\gamma (R\cup \{a\},d)$ for all remaining candidates, which involves building equivalence classes in $O(n·|R\left|\right)$ time. In the worst case with $|C_{\mathrm{sig}}|$ iterations, the total is $O\left(\right|C_{\mathrm{sig}}{|}^2·n)$. However, the MFDR-based stopping criterion typically limits the number of iterations well below $|C_{\mathrm{sig}}|$, and pre-screening often makes $|C_{\mathrm{sig}}|\ll m$. The overall time complexity is $O(n·m·r_{max}+|C_{\mathrm{sig}}{|}^2·n)$. In practice, the combination of statistical pre-screening and adaptive early stopping substantially reduces the number of dependency-degree evaluations compared to conventional greedy forward selection, which requires at most $O(m^2·n)$ operations.

5. Experiments

This section presents a comprehensive evaluation of the proposed MFDRRS algorithm to validate its effectiveness and efficiency. We first describe the experimental setup, including datasets, baseline methods, and evaluation metrics. Then, we report the results and analyze the performance of MFDRRS in comparison to existing rough-set-based attribute reduction methods and statistical based methods.

5.1. Experimental Setup

The experiments were conducted on a workstation with an AMD 3700X CPU, 32GB RAM, running Windows 10, ensuring a consistent experimental environment. To test the performance of the proposed MFDRRS algorithm, we selected six standard UCI datasets that vary in size, number of attributes, and class distributions, together with seven synthetic datasets designed to probe specific algorithmic properties under controlled conditions. Table 2 presents the characteristics of these datasets, including the number of objects, number of attributes, and the number of decision classes. The datasets were preprocessed to handle missing values and categorical attributes were encoded appropriately for the MFDR computation.

Besides, we construct seven controlled synthetic datasets to probe specific algorithmic properties of MFDRRS, each combining attributes of varying discriminative strength: deterministic attributes (perfectly determining the decision, $\gamma =1.0$), partial attributes (correlating with accuracy $\rho \in (0.70,0.80)$, yielding moderate MFDR), and noise attributes (uniform random, negligible MFDR $<0.01$). We also include structural variants such as redundant pairs and high-cardinality noise. Table 3 summarizes the composition of each dataset.

5.2. Compared Methods and Evaluation Metrics

The following methods were selected as baselines for comparison with the proposed MFDRRS method:

• Greedy Forward: Standard forward selection maximizing $\gamma$-gain[24].
• Greedy Backward: Standard backward elimination minimizing $\gamma$-loss[24].
• MI Ranking: Selects top-k attributes by $I(a_j;d)$ covering 90% of total mutual information[65].
• MaxRelevance: Greedily selects all attributes with non-negligible individual MI[66].
• JMI (Joint Mutual Information): Selects attributes maximizing $I(a_j;d)$ while penalizing redundancy with already-selected attributes[67].
• Exhaustive Search: Brute-force enumeration [24] of all subsets up to size n (feasible only for $n\le 20$).

To systematically assess the performance of MFDRRS and baseline methods, we employed three primary evaluation metrics:

• Number of selected attributes (s): Smaller is better for compactness.
• Dependency degree ($\gamma$): Higher is better for classification power.
• Runtime (t): Wall-clock time in seconds.

5.3. Results and Discussions

5.3.1. UCI Dataset Results

Table 4 presents the results of the MFDRRS method compared to baseline methods across the six UCI datasets. For each dataset, we report the number of selected attributes (s), the achieved dependency degree ($\gamma$), and the runtime in seconds (t). The best $\gamma$ values among methods with $s\le \mathrm{median}\left(s\right)$ are highlighted in bold.

As can be seen from Table 4, the proposed MFDRRS method achieves compact reductions on most UCI datasets. On breast-cancer, glass, wdbc, and wine, MFDRRS selects only 2 attributes achieving $\gamma =1.0$, matching Greedy Forward but avoiding Greedy Backward’s over-selection (30, 9, 30, and 13 attributes respectively). On ionosphere, MFDR selects 3 attributes with $\gamma =1.0$, identical to Greedy Forward but far more compact than Greedy Backward’s 34 attributes. And MFDRRS is conservative on large high-cardinality datasets. On the letter dataset (20,000 objects, 26 decision classes), MFDR selects only 6 attributes with $\gamma =0.82$ versus 16 for Greedy Forward ($\gamma =0.99$). This reflects the inherent difficulty of nominal attribute reduction when each attribute has many distinct values (letter recognition uses alphabetic characters as attribute values). The MFDR metric penalizes attributes whose class-wise scatter is low relative to within-class scatter, which is common when each value appears in multiple decision classes. Greedy Backward retains all attributes on every UCI dataset, failing to eliminate even clearly redundant ones. This confirms the well-known limitation that backward elimination is overly conservative in its removal criterion[53]. By design, MI-based methods do not output a dependency degree, so $\gamma$ is reported as 0.0 in the table. While MI provides an alternative measure of attribute importance, it does not directly quantify the rough-set classification power. MaxRelevance and MI Ranking tend to select many attributes (7–33 on UCI datasets), far exceeding MFDRRS’s selections. JMI selects 1–7 attributes—still more than MFDRRS on most datasets where FDR achieves full dependency. Furthermore, MFDRRS consistently outperforms Greedy Forward and Greedy Backward in runtime. For example, on the letter dataset, MFDRRS completes in 104.32 seconds versus 271.68 seconds for Greedy Forward and 53.77 seconds for Greedy Backward. The pre-screening step reduces the number of candidate attributes considered at each iteration, while the adaptive stopping criterion prevents unnecessary evaluations once the marginal gain falls below the threshold. This efficiency gain is particularly pronounced on larger datasets with many attributes. Besides, MFDR pre-filtering improves efficiency. On small-to-medium datasets ($m<500$), MFDRRS matches Greedy Forward in both selected attribute count and runtime. The MFDR-aware stopping rule rarely triggers on these datasets because the best candidate typically has sufficiently high MFDR. On the letter dataset, however, the MFDR filter reduces 16 candidates to just 2 significant attributes, enabling rapid termination. Finally, for small UCI datasets ($m<500$), all methods run in milliseconds to under one second. For larger datasets (letter, $m=20,000$), computation becomes expensive, with MFDRRS requiring about 104 seconds and Greedy Forward about 272 seconds due to the high cost of dependency degree computation on large equivalence classes.

Overall, MFDRRS yields the most compact reducts among all tested methods on every UCI dataset, while achieving full dependency ($\gamma =1.0$) on all small- to medium-sized datasets. On the letter dataset, MFDRRS achieves a notably more parsimonious reduction (6 attributes) with a slight trade-off in dependency—a compromise attributable to the extreme cardinality of its nominal attributes, which poses challenges for scatter-based discriminative measures. Across all benchmarks, MFDRRS outperforms Greedy Backward, MI Ranking, MaxRelevance, and JMI in compactness, and matches or exceeds Greedy Forward in quality, confirming that the Fisher discriminant ratio provides a reliable statistical ranking criterion that complements the rough-set framework.

5.4. Artificial Dataset Results

To further validate MFDRRS under controlled conditions, we evaluate on seven synthetic datasets (Table 3) designed to probe specific algorithmic properties: noise rejection, redundancy handling, high cardinality, and multi-class separability.

Table 5. Comparison of attribute reduction methods on UCI datasets.

Dataset	m	n	MFDRRS			Greedy Forward			Greedy Backward			MI Ranking			MaxRelevance			JMI			Exhaustive
			s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)
simple_separable	100	5	1	1.0000	0.001	1	1.0000	0.001	5	1.0000	0.001	2	0.0000	0.000	5	0.0000	0.000	1	0.0000	0.001	1	1.0000	0.000
with_noise	150	10	1	1.0000	0.003	1	1.0000	0.003	10	1.0000	0.003	5	0.0000	0.001	10	0.0000	0.000	1	0.0000	0.005	1	1.0000	0.001
redundant	120	8	1	1.0000	0.002	1	1.0000	0.002	8	1.0000	0.001	8	0.0000	0.001	8	0.0000	0.000	2	0.0000	0.004	1	1.0000	0.000
correlated	150	8	1	1.0000	0.003	1	1.0000	0.003	8	1.0000	0.002	4	0.0000	0.001	8	0.0000	0.000	1	0.0000	0.003	1	1.0000	0.000
high_cardinality	200	6	1	1.0000	0.004	1	1.0000	0.003	6	1.0000	0.003	5	0.0000	0.003	6	0.0000	0.003	3	0.0000	0.013	1	1.0000	0.001
multi_class	180	7	1	1.0000	0.004	1	1.0000	0.002	7	1.0000	0.003	5	0.0000	0.001	7	0.0000	0.001	2	0.0000	0.004	1	1.0000	0.001
imbalanced	150	6	1	1.0000	0.002	1	1.0000	0.001	6	1.0000	0.001	3	0.0000	0.000	6	0.0000	0.000	2	0.0000	0.002	1	1.0000	0.000

Table 5. Comparison of attribute reduction methods on UCI datasets.

Dataset	m	n	MFDRRS			Greedy Forward			Greedy Backward			MI Ranking			MaxRelevance			JMI			Exhaustive
			s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)	s	$\gamma$	t(s)
simple_separable	100	5	1	1.0000	0.001	1	1.0000	0.001	5	1.0000	0.001	2	0.0000	0.000	5	0.0000	0.000	1	0.0000	0.001	1	1.0000	0.000
with_noise	150	10	1	1.0000	0.003	1	1.0000	0.003	10	1.0000	0.003	5	0.0000	0.001	10	0.0000	0.000	1	0.0000	0.005	1	1.0000	0.001
redundant	120	8	1	1.0000	0.002	1	1.0000	0.002	8	1.0000	0.001	8	0.0000	0.001	8	0.0000	0.000	2	0.0000	0.004	1	1.0000	0.000
correlated	150	8	1	1.0000	0.003	1	1.0000	0.003	8	1.0000	0.002	4	0.0000	0.001	8	0.0000	0.000	1	0.0000	0.003	1	1.0000	0.000
high_cardinality	200	6	1	1.0000	0.004	1	1.0000	0.003	6	1.0000	0.003	5	0.0000	0.003	6	0.0000	0.003	3	0.0000	0.013	1	1.0000	0.001
multi_class	180	7	1	1.0000	0.004	1	1.0000	0.002	7	1.0000	0.003	5	0.0000	0.001	7	0.0000	0.001	2	0.0000	0.004	1	1.0000	0.001
imbalanced	150	6	1	1.0000	0.002	1	1.0000	0.001	6	1.0000	0.001	3	0.0000	0.000	6	0.0000	0.000	2	0.0000	0.002	1	1.0000	0.000

On every synthetic dataset, MFDRRS selects exactly 1 attribute with $\gamma =1.0$, matching Greedy Forward and Exhaustive Search while far outperforming Greedy Backward (5–10 attributes), MI Ranking (2–8), and MaxRelevance (5–10). MFDR assigns scores of ${10}^{10}$ to deterministic attributes—attribute 0 (a bijective permutation of the class) and their exact or near-exact copies—while noise attributes receive near-zero MFDR ($<0.02$), and partially deterministic attributes (80% accuracy) fall in the intermediate range (0.49–0.83), cleanly separating informative from uninformative features. On the redundant dataset (8 attributes forming 4 identical pairs), all attributes individually achieve $\gamma =1.0$, yet MFDR retains only 1 of 8, whereas Greedy Backward keeps all 8 and MI-based methods select 2–8, confirming that MFDR avoids redundancy accumulation. On high-cardinality data (up to 50 distinct values per attribute), MFDR selects 1 attribute with $\gamma =1.0$, while MI Ranking and MaxRelevance select 5 and 6 respectively, demonstrating that the between-class/within-class scatter formulation naturally accounts for cardinality without explicit normalization. On the imbalanced (80/80/40 split, 3 classes) and multi_class (5 classes) datasets, MFDRRS again selects 1 attribute with full dependency, while JMI achieves 1–2 attributes but with $\gamma =0.0$, indicating an invalid rough-set reduct. Computationally, all methods run in milliseconds ($m\le 200$); MFDRRS’s runtime (0.002–0.005 s) is comparable to Greedy Forward and within an order of magnitude of the fastest baselines, with the MFDR computation adding negligible overhead.

Collectively, the synthetic results demonstrate that MFDRRS reliably achieves the optimal reduct size ($s=1$) across a range of data complexities—noise, redundancy, high cardinality, imbalance, and multi-class separability—without sacrificing dependency ($\gamma =1.0$). The distinct MFDR score gap between relevant and irrelevant attributes substantiates the pre-filtering mechanism, while the low computational cost ensures practical scalability.

6. Conclusions

MFDRRS provides a principled statistical criterion for attribute ranking: the Fisher discriminant ratio offers a transparent and interpretable measure of attribute discriminability that complements the rough-set dependency degree. Unlike heuristic methods that rely solely on marginal gains in $\gamma$, MFDRRS employs a well-established statistic that captures both between-class separation and within-class compactness. The MFDR-aware stopping rule eliminates manual threshold calibration, enhancing practical usability over competing approaches. The pre-filtering step further reduces the search space prior to greedy selection, thereby improving computational efficiency.

Future research will pursue these directions: (1) extending FDR to mixed-type (nominal and numeric) data; (2) developing multi-attribute FDR measures to capture joint attribute interactions; (3) scaling the framework to large-scale datasets via sparse approximation and distributed computation. Finally, we plan to validate the method in broader application domains, including text categorization and bioinformatics, beyond tabular classification.