Dual-Channel Attention and Category-Guided Distillation for Lightweight Plant Disease Recognition

1. Introduction

Accurate plant disease recognition is important for crop management and food security. Early and reliable diagnosis can reduce crop losses, improve pesticide efficiency, and support precision agriculture workflows [1]. Traditional image-analysis approaches rely on hand-crafted color, texture, and shape descriptors. Although these methods are computationally simple, their discriminative ability is highly sensitive to environmental noise and feature design [2].

Deep learning has improved plant disease recognition by learning hierarchical visual representations directly from data [3]. In particular, CNNs have achieved strong performance on benchmark datasets, but their local receptive fields limit the modeling of long-range dependencies and global contextual relationships [4]. This issue is particularly important for leaf disease recognition, where symptoms may appear as distributed lesions, irregular discoloration, or fine-grained texture changes over a large spatial extent. For instance, tomato diseases such as early blight and late blight display visually overlapping characteristics that are difficult to distinguish using purely localized features, as illustrated in Figure 1. This motivates models that can integrate both local and global contextual relationships.

Transformers provide a promising alternative because self-attention can capture long-range dependencies and global context more explicitly than convolutions [5]. However, existing visual Transformer models generally emphasize either spatial interactions or token-level global reasoning, while the complementarity between spatial localization and channel-wise semantic selection is less explicitly modeled. In plant disease recognition, both aspects are important: spatial attention can focus on lesion regions, while channel attention can amplify disease-relevant spectral and texture responses. A model that jointly exploits these two dimensions is therefore desirable.

Another practical challenge is deployment. High-capacity Transformer models are often too expensive for mobile agricultural devices, drones, or edge terminals. Knowledge distillation (KD) offers an effective way to compress a powerful teacher into a lightweight student. Nevertheless, conventional KD typically transfers only softened output probabilities, which are insufficient for fine-grained disease recognition because valuable intermediate feature relationships are not fully preserved.

Motivated by these observations, this paper proposes a dual-channel attention network with category-guided knowledge distillation for plant disease recognition. The framework consists of a Dual-Channel Feature Enhancement Network (DC-FEN) as the teacher and a Category-Guided Knowledge Distillation (CGKD) strategy for transferring knowledge to a lightweight MobileNetV3 [6] student. The proposed design aims to strengthen both discriminative representation learning and efficient deployment.

The main contributions are summarized as follows:

1.

We design a Dual-Channel Feature Enhancement Network (DC-FEN) that combines spatial attention and channel attention to capture complementary lesion localization cues and channel-level semantics for plant disease recognition.

2.

We propose a Category-Guided Knowledge Distillation (CGKD) framework that jointly transfers output-level category relationships and intermediate feature-level relational knowledge from the teacher to the student.

3.

We provide comparative experiments and ablation studies showing that the proposed framework improves a lightweight student network while maintaining competitive recognition performance on Plant Pathology 2021 (FGVC8) dataset and a challenging tomato disease benchmark derived from the PlantVillage dataset.

2. Related Work

This section reviews prior work from four perspectives: (1) Traditional hand-crafted feature methods, (2) CNN-based deep learning methods, (3) Transformer and CNN–Transformer visual recognition models, and (4) Knowledge Distillation methods for efficient deployment.

2.1. Traditional Methods

Early plant disease recognition methods primarily relied on manually designed feature extraction such as color histograms, texture features, and shape measurements [7]. These descriptors were then used as inputs for standard classifiers [8]. However, such methods are sensitive to variations in image quality and complex background clutter, which limits their practical use in field scenarios.

Color and texture represent fundamental visual attributes of plant leaves. Early research frequently utilized color histograms and texture descriptors for recognition tasks. For instance, Bhimte et al. successfully identified various plant species by extracting leaf color distributions and texture features, coupled with a Support Vector Machine (SVM) classifier [9]. Similarly, texture-oriented feature extractors-most notably GLCM and LBP have been widely applied to capture fine-grained surface patterns associated with specific disease symptoms or plant species [10].

To address the limitations of single-feature representations, multi-feature fusion strategies have been proposed. Lv et al. combined improved LBP, Histogram of Oriented Gradients, and color features, followed by dimensionality reduction and classification using an Extreme Learning Machine, achieving improved performance on benchmark datasets [11]. Despite their effectiveness, such fusion-based methods increase computational complexity and remain heavily dependent on manual feature engineering, motivating a gradual transition toward deep learning-based solutions [12].

2.2. Deep Learning-Based Methods

The rapid maturation of deep learning has fundamentally shifted the paradigm of plant disease analysis toward data-driven frameworks. In contrast to conventional machine learning approaches that depend largely on manually designed features, deep neural networks can learn hierarchical representations directly from raw image data. This capability enables more robust and accurate recognition performance under varying imaging environments and complex visual conditions. Recent studies have emphasized the importance of contextual information modeling in overcoming difficulties caused by complicated backgrounds, subtle symptom variations, and high visual similarity among different disease categories [13,14,15].

Early deep learning studies primarily employed convolutional neural networks (CNNs) combined with transfer learning. Kumar et al. demonstrated that fine-tuned deep CNNs, including AlexNet, GoogLeNet, and ResNet, achieved high accuracy for tomato disease classification on the PlantVillage dataset, with ResNet optimized via Stochastic Gradient Descent yielding the best performance [16]. Similarly, Agarwal et al. utilized lightweight CNN architectures such as AlexNet and SqueezeNet for multi-class tomato disease recognition, achieving competitive results while reducing computational cost [17]. Building upon these efforts, subsequent works have refined CNN architectures to improve feature expressiveness and training stability. For instance, residual learning mechanisms have been shown to alleviate vanishing gradient issues and facilitate the extraction of complex disease patterns, thereby enhancing classification performance in large-scale plant disease datasets [18,19].

One inherent drawback of CNNs is their limited receptive field, which constrains their capacity to effectively capture long-range dependencies. Plant disease symptoms often appear as small, spatially distributed lesions or irregular patterns across the leaf surface. As a result, CNNs may fail to capture global contextual relationships among dispersed disease cues, leading to ambiguous or incorrect predictions [20]. To mitigate this issue, recent research has explored architectural enhancements aimed at enriching contextual representation [4,21].

Multi-scale learning strategies have been widely adopted to address this challenge. Elfatimi et al. proposed a multi-scale CNN framework that aggregates features across different resolutions, enabling more reliable discrimination of disease regions under complex backgrounds [22]. Similarly, Feature Pyramid Networks (FPNs) have been integrated with deep CNNs to construct hierarchical feature fusion architectures, improving the representation of fine-grained disease patterns across spatial scales [23]. While these methods enhance contextual perception to some extent, they still rely on convolutional inductive biases and remain limited in modeling global dependencies explicitly.

2.3. Applications of Transformers in Visual Recognition

Originally designed for natural language processing tasks, transformer architectures have recently attracted considerable attention in visual recognition, largely due to their self-attention mechanism that enables effective modeling of long-range dependencies. In contrast to CNNs, Transformers capture global contextual relationships directly, making them particularly suitable for fine-grained visual analysis. Vision Transformer (ViT) and its hierarchical variants, such as the Swin Transformer, have demonstrated competitive or superior performance across diverse computer vision benchmarks [24].

In plant disease recognition, Transformers have shown promising potential for modeling complex spatial dependencies among disease symptoms. Zhang et al. adapted the Swin Transformer for rice blast disease classification, achieving strong performance by leveraging hierarchical self-attention to capture multi-scale contextual information [25]. Despite these advantages, pure Transformer-based models typically treat spatial and channel dimensions independently, which may limit their discriminative power when inter-class visual differences are subtle and highly correlated [26].

To overcome this limitation, hybrid CNN-Transformer architectures have been proposed. Models such as the Convolutional Vision Transformer (CvT) integrate convolutional layers to encode local spatial inductive biases before applying Transformer blocks for global reasoning. These hybrid approaches often achieve improved performance and faster convergence compared to standalone Transformers [27,28]. However, most existing hybrid designs rely on heuristic fusion strategies that inadequately model dynamic interactions between spatial, channel, and semantic dimensions [29]. As a result, their ability to exploit complementary feature relationships remains constrained, particularly in complex agricultural scenarios involving diverse plant species and disease manifestations.

2.4. Knowledge Distillation Methods

Knowledge distillation (KD) has emerged as an effective learning paradigm for enhancing both the efficiency and generalization ability of deep neural networks. This is achieved by transferring informative knowledge from a large, high-capacity teacher model to a lightweight student model with fewer parameters. Recent studies emphasize that effective distillation requires not only matching output predictions but also carefully aligning intermediate representations and learning dynamics between teacher and student networks [30,31].

Theoretical and empirical investigations in recent years have deepened the understanding of KD mechanisms. Phuong and Lampert established convergence and generalization bounds for distilled student models, highlighting the influence of optimization bias and data geometry on distillation performance. Subsequent empirical analyses demonstrated that excessively large teachers may hinder student learning due to representational mismatch, motivating the design of teacher–assistant or capacity-aware distillation frameworks [32]. These findings suggest that distillation effectiveness depends critically on the compatibility between teacher and student architectures rather than teacher size alone.

Beyond conventional convolutional networks, knowledge distillation has been extended to structured and relational learning paradigms. Graph Knowledge Distillation (GKD) has recently emerged as an effective approach for compressing Graph Neural Networks (GNNs) while preserving relational information. Representative works distill structural knowledge from large GNNs into compact GNNs or even multilayer perceptrons, significantly reducing inference cost and latency [33,34]. Such methods demonstrate that relational knowledge can be implicitly encoded without explicit graph propagation during inference.

Within the specific context of plant disease recognition, KD has been widely introduced as a strategy to achieve a trade-off between high predictive accuracy and practical deployment efficiency. Recent studies have shown that distilling knowledge from high-capacity detectors or classifiers into lightweight student models can achieve near-teacher-level performance under real-world constraints [35]. However, most existing approaches focus on prediction-level supervision and overlook structured relationships among disease patterns, plant species, and contextual cues, leaving significant room for more expressive distillation strategies.

3. Method

In this study, we introduce a plant disease recognition framework in which a dual-channel attention-based network is designed as the teacher model, while MobileNetV3 is employed as the lightweight student network. The teacher model captures spatial and channel-wise contextual dependencies through a dual-channel self-attention mechanism. The student network is optimized not only using ground-truth labels but also under the supervision of the teacher model. We further use a category-guided knowledge distillation strategy to explicitly transfer inter-class feature relationships from the teacher, thereby narrowing the representation gap observed when MobileNetV3 is trained independently.

3.1. Dual-Channel Feature Enhancement Network

This section presents the Dual-Channel Feature Enhancement Network (DC-FEN), as shown in Figure 2, which uses a dual-channel attention mechanism to refine feature representations. The network extracts spatial context and channel-wise dependencies, enabling more discriminative representation learning for plant disease classification. DC-FEN is composed of several Dual-Channel Attention (DC-A) blocks. Each block uses spatial and channel attention in parallel, so that the model can capture complementary dependencies in the input features.

Let the input image be defined as $I\in {\mathbb{R}}^{H\times W\times C}$, where H, W, and C denote the height, width, and number of channels, respectively. The Dual-Channel Attention strategy incorporates two complementary modules: spatial attention and channel attention. These mechanisms generate attention maps that facilitate focusing on the most salient regions of the feature maps while emphasizing discriminative channels critical for accurate classification.

Spatial Attention. Spatial attention aims to determine where disease-related regions are located. In plant disease recognition, this capability is particularly important because discriminative symptoms are often spatially localized and irregularly distributed. For example, early blight, late blight, rust, chlorosis, and mildew may appear as scattered lesions, blurred boundaries, or non-uniform texture changes over the leaf surface. Purely local convolutional operations may fail to capture the long-range dependencies among these spatially separated disease cues. Therefore, spatial attention is introduced to explicitly model global interactions among different spatial locations and to enhance lesion-sensitive regions while suppressing irrelevant background interference.

Given an input feature map $F\in {\mathbb{R}}^{H\times W\times C}$, we first reshape it into a token sequence

$$F^{\prime }\in {\mathbb{R}}^{N\times C},N=H\times W,$$

(1)

where H, W, and C denote the height, width, and channel dimension of the feature map, respectively, and N is the number of spatial tokens. We then project $F^{\prime }$ into query, key, and value representations:

$$Q=F^{\prime }{W}_Q,K=F^{\prime }{W}_K,V=F^{\prime }{W}_V,$$

(2)

where $Q,K,V\in {\mathbb{R}}^{N\times d}$, and d denotes the embedding dimension.

The spatial attention matrix is computed by scaled dot-product attention:

$$A_{\mathrm{spa}}=softmax\left({\displaystyle \frac{Q{K}^{\top }}{\sqrt{d}}}\right)\in {\mathbb{R}}^{N\times N}.$$

(3)

Here, each element of $A_{\mathrm{spa}}$ measures the relevance between two spatial positions, allowing each token to aggregate contextual information from all other positions. In this way, the model can establish long-range dependencies across spatially distributed disease regions, thereby strengthening its ability to represent lesion-associated structures more effectively and comprehensively.

The spatially enhanced feature is then obtained by applying the attention matrix to the value representation:

$$F_{\mathrm{spa}}=A_{\mathrm{spa}}V\in {\mathbb{R}}^{N\times d}.$$

(4)

This operation generates a new spatial feature representation in which each location is enriched by global contextual cues from other disease-relevant regions.

After that, $F_{\mathrm{spa}}$ is reshaped back to the spatial form $H\times W\times d$. To further refine the lesion localization effect, a $1\times 1$ convolution followed by sigmoid activation is used to produce a spatial gating map:

$$G_{\mathrm{spa}}=\sigma \left({Conv}_{1\times 1}\left(F_{\mathrm{spa}}\right)\right),$$

(5)

where $\sigma (·)$ denotes the sigmoid activation function.

The produced gating map assigns higher responses to disease-sensitive spatial regions and lower responses to background or less informative areas. Consequently, the network is guided to focus more strongly on key discriminative cues such as lesion locations, discoloration regions, and abnormal structural patterns in leaves that are essential for distinguishing different disease categories.

Channel Attention. While spatial attention focuses on where disease-related regions are located, channel attention aims to determine which feature channels are most discriminative for plant disease recognition. For example, lesion-type diseases may be characterized more strongly by texture-sensitive channels, while chlorosis or nutrient-deficiency-like symptoms may be more distinguishable in color- or intensity-related channels. Therefore, channel attention is introduced to adaptively model inter-channel dependencies and to emphasize those channels that are most relevant to disease discrimination.

Given the same input feature map $F\in {\mathbb{R}}^{H\times W\times C}$, we first reshape it into a token sequence

$$F^{\prime }\in {\mathbb{R}}^{N\times C},N=H\times W,$$

(6)

and then project it into query, key, and value representations:

$$Q=F^{\prime }{W}_Q,K=F^{\prime }{W}_K,V=F^{\prime }{W}_V,$$

(7)

where $Q,K,V\in {\mathbb{R}}^{N\times d}$, and d denotes the embedding dimension.

To model the relationships among channels, we transpose the projected query and compute channel-wise attention as

$$A_{\mathrm{cha}}=softmax\left({\displaystyle \frac{Q^{\top }K}{\sqrt{N}}}\right)\in {\mathbb{R}}^{d\times d}.$$

(8)

Here, $A_{\mathrm{cha}}$ captures the dependency structure between channels, and the normalization term uses N because the channel-wise dot product is accumulated over spatial tokens.

The channel-enhanced feature is then obtained by

$$F_{\mathrm{cha}}=V{A}_{\mathrm{cha}}\in {\mathbb{R}}^{N\times d}.$$

(9)

Through this operation, each channel is re-weighted according to its interaction with other channels, resulting in a feature representation that better highlights disease-relevant spectral, color, and texture characteristics.

After reshaping $F_{\mathrm{cha}}$ back to the $H\times W\times d$ form, a $1\times 1$ convolution followed by sigmoid activation is used to generate a channel gating map:

$$G_{\mathrm{cha}}=\sigma \left({Conv}_{1\times 1}\left(F_{\mathrm{cha}}\right)\right).$$

(10)

This gate adaptively enhances informative disease-related channels and suppresses less relevant responses, thereby improving the discriminability of subtle plant disease patterns.

Finally, the spatially enhanced and channel-enhanced representations are fused through residual gated aggregation:

$$F_{\mathrm{out}}=F_d+G_{\mathrm{spa}}\odot F_{\mathrm{spa}}+G_{\mathrm{cha}}\odot F_{\mathrm{cha}},$$

(11)

where $F_d$ denotes the input feature projected to the same dimension as $F_{\mathrm{spa}}$ and $F_{\mathrm{cha}}$, and ⊙ denotes element-wise multiplication. All terms are reshaped to $H\times W\times d$, and the gating maps are broadcast when necessary. This fusion preserves the original representation while adaptively incorporating spatial and channel cues.

3.2. Category-Guided Knowledge Distillation

Although lightweight models are more suitable for real-world deployment, their limited representation capacity often makes it difficult to capture the subtle inter-class differences among visually similar plant diseases. To address this limitation, we propose a Category-Guided Knowledge Distillation (CGKD) framework that enables the transfer of both category-level semantic information and feature-level relational cues from the teacher model to a lightweight student network. The complete architecture of the proposed method is presented in Figure 3. In our framework, the proposed Dual-Channel Feature Enhancement Network (DC-FEN) serves as the teacher network, while MobileNetV3 is adopted as the student network.

The proposed CGKD framework contains two complementary components. The first component, referred to as Category Knowledge Distillation (CKD), transfers category-level semantic information by aligning the softened prediction distributions of the teacher and student. This enables the student to learn inter-class similarity structure beyond hard labels. The second component, referred to as Feature Knowledge Distillation (FKD), transfers fine-grained feature knowledge from intermediate representations, so that the student can better preserve discriminative disease patterns learned by the teacher.

The overall training objective is defined as

$${\mathcal{L}}_{\mathrm{total}}={\mathcal{L}}_{\mathrm{cls}}+{\lambda }_{\mathrm{CKD}}{\mathcal{L}}_{\mathrm{CKD}}+{\lambda }_{\mathrm{FKD}}{\mathcal{L}}_{\mathrm{FKD}},$$

(12)

where ${\mathcal{L}}_{\mathrm{cls}}$ denotes the supervised classification loss, and ${\lambda }_{\mathrm{CKD}}$ and ${\lambda }_{\mathrm{FKD}}$ are trade-off coefficients used to balance the category-level and feature-level distillation losses, respectively. For single-label experiments, ${\mathcal{L}}_{\mathrm{cls}}$ is implemented as cross-entropy loss with softmax outputs. For the multi-label FGVC8 setting, the final activation is changed to sigmoid and ${\mathcal{L}}_{\mathrm{cls}}$ is implemented as binary cross-entropy loss.

Category Knowledge Distillation (CKD). To transfer category-level semantic knowledge, we perform alignment between the teacher and student by matching their softened prediction distributions. Let $z^t$ and $z^s$ denote the logits produced by the final prediction layers of the teacher and student before activation, respectively. For single-label classification, their softened probability distributions are computed as

$$p_k^t(x_i;\tau )={\displaystyle \frac{exp\left(z_k^t\left(x_i\right)/\tau \right)}{{\sum }_{m=1}^{K}exp\left(z_m^t\left(x_i\right)/\tau \right)}},$$

(13)

$$p_k^s(x_i;\tau )={\displaystyle \frac{exp\left(z_k^s\left(x_i\right)/\tau \right)}{{\sum }_{m=1}^{K}exp\left(z_m^s\left(x_i\right)/\tau \right)}},$$

(14)

where K denotes the number of disease categories and $\tau$ denotes the temperature parameter. A larger $\tau$ produces a softer probability distribution, which helps reveal inter-class similarity information. For notation consistency, $\tau$ is used only for temperature throughout this paper, while the superscript t denotes the teacher model.

The CKD loss is formulated as

$${\mathcal{L}}_{\mathrm{CKD}}=-{\displaystyle \frac{{\tau }^2}{B}}{\displaystyle \sum _{i=1}^B}{\displaystyle \sum _{k=1}^K}{p}_k^t(x_i;\tau )log{p}_k^s(x_i;\tau ),$$

(15)

where B denotes the mini-batch size. This loss encourages the student network to mimic the category-level semantic structure encoded in the teacher predictions. In the FGVC8 multi-label experiment, the same alignment is applied to sigmoid-based teacher and student confidence scores rather than softmax-normalized probabilities. Specifically, we define

$$q_k^t(x_i;\tau )=\sigma \left(z_k^t\left(x_i\right)/\tau \right),q_k^s(x_i;\tau )=\sigma \left(z_k^s\left(x_i\right)/\tau \right),$$

(16)

and use the multi-label CKD loss

$${\mathcal{L}}_{\mathrm{CKD}}^{\mathrm{ml}}=-{\displaystyle \frac{1}{BK}}{\displaystyle \sum _{i=1}^B}{\displaystyle \sum _{k=1}^K}\left[q_k^{t}log{q}_k^s+\left(1-q_k^t\right)log\left(1-q_k^s\right)\right].$$

(17)

For FGVC8, ${\mathcal{L}}_{\mathrm{CKD}}$ in the overall objective is replaced by ${\mathcal{L}}_{\mathrm{CKD}}^{\mathrm{ml}}$.

Feature Knowledge Distillation (FKD). In addition to output-level supervision, we further transfer feature-level knowledge from an intermediate representation of the teacher to the student. Specifically, let $F^t\in {\mathbb{R}}^{N\times d_t}$ and $F^s\in {\mathbb{R}}^{N\times d_s}$ denote the intermediate token features extracted from the teacher and student, respectively, before the final classification head. Here, N is the number of spatial tokens and $d_t$, $d_s$ are the corresponding feature dimensions. Since the teacher and student may have different feature dimensions, a learnable linear projection $g(·)$ is introduced to map the student features into the teacher feature space. This operation is necessary because the teacher and student not only differ in feature dimensionality but also exhibit different representation spaces due to their heterogeneous architectures.

$${\tilde{F}}^s=g\left(F^s\right)\in {\mathbb{R}}^{N\times d_t}.$$

(18)

The direct feature alignment loss is then defined as

$${\mathcal{L}}_{\mathrm{fea}}={\displaystyle \frac{1}{N{d}_t}}{∥F^t-{\tilde{F}}^s∥}_2^2.$$

(19)

This term helps the student preserve fine-grained disease-related feature patterns learned by the teacher.

To further transfer structural relationships among tokens, we construct token correlation matrices for the teacher and student features:

$$M^t={\displaystyle \frac{F^t{\left(F^t\right)}^{\top }}{\sqrt{d_t}}},M^s={\displaystyle \frac{{\tilde{F}}^s{\left({\tilde{F}}^s\right)}^{\top }}{\sqrt{d_t}}},$$

(20)

where $M^t,M^s\in {\mathbb{R}}^{N\times N}$ describe pairwise relationships among spatial tokens. Here, a token corresponds to one spatial location obtained after flattening the feature map.

The relational distillation loss is defined as

$${\mathcal{L}}_{\mathrm{rel}}={\displaystyle \frac{1}{N^2}}{∥M^t-M^s∥}_2^2.$$

(21)

Finally, the feature-level distillation loss is written as

$${\mathcal{L}}_{\mathrm{FKD}}={\mathcal{L}}_{\mathrm{fea}}+{\mathcal{L}}_{\mathrm{rel}}.$$

(22)

By jointly optimizing the above objectives, the proposed CGKD framework allows the lightweight MobileNetV3 student to inherit both category-level semantic knowledge and fine-grained feature relationships from the DC-FEN teacher and improves lightweight plant disease recognition performance.

4. Experiments

In this section, we begin by presenting the experimental setup of the proposed method, including details of the dataset, teacher and student model configuration, and training details. Then we evaluate the proposed framework against several representative baseline methods on the PlantVillage dataset [36] and further evaluate it on the more challenging Plant Pathology 2021 (FGVC8) dataset [37]. Finally, we perform ablation studies to systematically evaluate and validate the contributions of the proposed attention design and the knowledge distillation strategy.

4.1. Experimental Setup

We evaluate the proposed method on the PlantVillage dataset, which contains 54,305 images covering 38 classes from 14 crop species. The subset of tomato images consists of 16,415 images of diseased leaf covering nine distinct disease categories, along with 1,591 images of healthy leaves. Because the PlantVillage dataset lacks complex backgrounds and many existing methods achieve near-perfect performance on it, we also include the FGVC8 dataset to test the proposed framework under more realistic conditions. For the controlled PlantVillage comparison, all generic baselines in Table 2 are trained from scratch under the same 70/15/15 train/validation/test split, input resolution of $224\times 224$, and data augmentation protocol, without ImageNet pretraining unless explicitly stated. This from-scratch setting is used to isolate architecture and distillation effects rather than to maximize the absolute PlantVillage performance.

4.1.1. Teacher and Student Models

The teacher model is the proposed Dual-Channel Feature Enhancement Network (DC-FEN), while the student model is MobileNetV3. DC-FEN is designed to strengthen both spatial lesion localization and channel-wise semantic discrimination through the Dual-Channel Attention Block (DCAB), whereas MobileNetV3 is selected as a lightweight backbone suitable for efficient deployment on resource-constrained devices.

The convolutional backbone contains five stages composed of depthwise separable convolutions and residual structures. The output feature dimensions of the five stages are $[16,112,112]$, $[16,56,56]$, $[24,28,28]$, $[40,14,14]$, and $[40,14,14]$, where each tuple denotes the channel number, height, and width, respectively. To enrich contextual representation without incurring prohibitive computational costs, two DCAB modules are embedded after the fourth and fifth stages of the teacher network. Each DCAB uses an embedding dimension of $d=40$ and single-head spatial and channel attention. The classification head is composed of a global average pooling layer, which is followed by a fully connected layer used to generate the final prediction output. During knowledge distillation, intermediate teacher features extracted from the enhanced stages are aligned with the corresponding student features after projection, which is consistent with the technical design described in Section 3.2.

4.1.2. Training Details

Following the data augmentation strategy used in MobileNetV3 [6], the training images are augmented by random rotation in the range of $[-\pi /4,+\pi /4]$, random scaling within $[0.95,1.05]$, and random horizontal flipping. For the loss function, $\tau$ is set as 1, ${\lambda }_{\mathrm{CKD}}$ and ${\lambda }_{\mathrm{FKD}}$ are set as 0.2. We use $\tau =1$ to avoid over-smoothing the teacher predictions, since fine-grained plant disease recognition requires preserving subtle inter-class confidence differences. In this setting, the CKD loss transfers the teacher’s category-level distribution directly, which provides stable supervision for the lightweight student model.

During training, we use the adam_onecycle optimization strategy, setting the initial learning rate to 0.001, momentum to 0.9, and weight decay to 0.01. The models are trained for 100 epochs using a batch size of 64 on two NVIDIA RTX 3090 GPUs. To reduce overfitting, a Dropout layer with a rate of 0.2 is incorporated. In addition, a learning rate scheduler is applied to dynamically adjust the learning rate throughout training. Unless otherwise specified, the remaining implementation details follow the standard OpenMMLab protocol.

4.1.3. Model Complexity and Inference Efficiency

To further evaluate deployment efficiency, we report the parameter count, FLOPs, and inference time in Table 1. FLOPs are calculated using an input resolution of $224\times 224$, while inference time is evaluated on an NVIDIA RTX 3090 GPU with batch size 1 under the same input resolution. The reported inference time is averaged over multiple runs after warm-up. Since CGKD is used only during training, the deployed model is the distilled MobileNetV3 student and introduces no additional inference parameters or FLOPs over the MobileNetV3 backbone.

4.2. Results

We evaluate the proposed method from three perspectives. First, we compare it with representative baseline models on the PlantVillage dataset under the above fine-grained setting. Then, we further evaluate the proposed method on the FGVC8 dataset. Finally, we analyze the effectiveness of the proposed CKD and FKD components on different backbone networks and provide confidence and visualization analyses.

4.2.1. Comparison on the PlantVillage Dataset

We first compare the proposed method with representative generic baseline models under the same experimental setting. The compared models include three groups: (1) conventional CNNs, including Inception-v3, VGG-16, VGG-19, and ResNet; (2) lightweight models, including ShuffleNetV2 and MobileViT; and (3) strong modern backbones, including EfficientNet, ViT, ResNeSt, and EfficientFormer. These baselines are selected to cover classical convolutional models, efficient lightweight architectures, and recent Transformer-based or hybrid recognition paradigms.

As shown in Table 2, under this from-scratch controlled protocol, the proposed distilled MobileNetV3 student achieves 86.25%, outperforming the compared generic backbones trained under identical conditions. Traditional architectures, including Inception_v3, VGG-16, VGG-19, and ResNet, obtain accuracies between 69.56% and 75.12% in this setting. Lightweight models such as ShuffleNet_v2 and MobileViT improve the performance to 78.24% and 79.75%, respectively. Models including EfficientNet, ViT, ResNeSt, and EfficientFormer further improve the accuracy to the range of 80.40%–83.89%. This comparison evaluates the effectiveness of the distillation framework under a unified from-scratch setting; it is not intended to compete with state-of-the-art PlantVillage results obtained via transfer learning or specialized preprocessing, which can exceed 99% as listed in Table 3. The PlantVillage experiment, therefore, serves as a controlled ablation of the distillation strategy rather than as the primary accuracy benchmark.

It should be noted that Table 2 reports only the results reproduced under our unified experimental setting. Recent plant-disease-specific methods reported in the literature are discussed separately, since their results are often obtained under different dataset subsets, splits, preprocessing strategies, or transfer-learning settings.

In addition to the generic baselines in Table 2, recent studies have also proposed plant-disease-specific models, such as Deep Multi-scale Convolutional Neural Network (DMCNN) and transformer-based transfer-learning methods, as summarized in Table 3. These methods are included to broaden the coverage of recent disease-oriented designs rather than to claim strict protocol-level fairness, because their reported results are obtained under different experimental settings.

4.2.2. Comparison on FGVC8

FGVC8 contains field-collected leaf images with more complex backgrounds, larger appearance variations, and more realistic disease patterns than PlantVillage. To further evaluate the practical relevance of our method, we additionally compare our results with recent methods.

For FGVC8, we follow the official multi-label setting. The model outputs sigmoid confidence scores for each label, and a fixed threshold of 0.5 is used to obtain binary predictions. The mean F1-score is computed over all disease labels. All images are resized to $224\times 224$, and the same training strategy as described above is used unless otherwise specified.

Since FGVC8 is a multi-label benchmark, mean F1-score is adopted as the primary evaluation metric. As shown in Table 4, our method achieves a mean F1-score of 96.48%, which is competitive with recent literature-reported methods. Although CTPlantNet reports a higher mean F1-score of 97.57%, it is evaluated under a different experimental protocol. Therefore, this comparison is used only as supplementary evidence rather than a strict protocol-level ranking.

4.2.3. Effect of CKD and FKD on Different Backbones

We further perform additional experiments to evaluate the effectiveness of Feature Knowledge Distillation (FKD) and Category Knowledge Distillation (CKD) on other established architectures. All experiments are performed under the same data partitioning and training settings to ensure consistency and fair comparison.

As shown in Table 5, VGG16 + FKD reaches 82.71%, indicating that feature-level distillation improves the representation quality of the baseline network. VGG16 + CKD also improves the performance to 80.24%, but the gain is smaller than that of FKD alone. The joint version, VGG16 + FKD + CKD, achieves the best result of 85.36%, suggesting that the two types of knowledge are complementary.

A similar trend is observed for ResNet, as shown in Table 5. ResNet + FKD reaches 83.39%, higher than ResNet + CKD at 82.21%. The joint model achieves 86.16%, again showing that category-level and feature-level knowledge provide complementary benefits.

An important observation from Table 5 is that the distilled MobileNetV3 student achieves performance that is comparable to, and in some cases even surpasses, that of significantly larger and more computationally expensive distilled backbone models. This result indicates that the proposed CKD+FKD framework can transfer useful knowledge to a lightweight architecture.

Beyond overall classification accuracy, it is crucial to evaluate the certainty and discriminative power of the proposed model when distinguishing between diverse disease categories. To this end, we analyze the prediction confidence of the distilled MobileNetV3 student model across selected plant species and disease manifestations. As presented in Table 6, the displayed sigmoid confidence scores provide a qualitative illustration of the prediction responses over selected disease categories under the multi-label visualization setting. Since only representative output categories are shown, zero-valued entries should not be interpreted as the complete absence of confidence over the full label space. Instead, the table is intended to show that the distilled student generally produces concentrated prediction responses for representative samples, suggesting that the CKD strategy helps transfer discriminative category-level information from the teacher network.

4.3. Visualization

In this experiment, we compare the feature maps extracted by three models: the base model, the teacher model, and the base model after distillation.

As illustrated in Figure 4, the original images and feature maps extracted by the base model, teacher model, and distilled student model are compared. The base model tends to focus on low-level details, whereas the teacher model presents clearer boundaries and more discriminative patterns. The distilled student model produces feature maps that are closer to those of the teacher, suggesting that the distillation process improves the representation of disease-related structures.

The comparison suggests that distillation improves the feature extraction capability of the base model and makes its feature maps closer to those of the teacher model.

4.4. Ablation Study

We performed an ablation study to evaluate the contributions of different components in our model.

The ablation study shows that the standalone teacher model yields the highest accuracy of 87.35%, as shown in Table 7. The MobileNetV3 student after distillation attains a competitive accuracy of 86.25%. In contrast, the Baseline MobileNetV3 achieves 75.88%, showing the gain brought by the proposed distillation strategy. Throughout this paper, “Ours” refers to the deployed distilled MobileNetV3 student, which is the lightweight model intended for practical deployment; the teacher DC-FEN serves only as a knowledge source during training.

Ablation Study on FKD and CKD. In this section, we evaluate the contributions of Feature Knowledge Distillation (FKD) and Category Knowledge Distillation (CKD) to the performance of our plant disease classification model on the PlantVillage dataset. Specifically, experiments were conducted to assess the individual and combined effects of FKD and CKD, using the teacher model as the comparative baseline.

Table 7 shows the effect of FKD and CKD on model performance. When FKD is applied, the student model reaches an accuracy of 85.43%, indicating that feature-level distillation helps the student extract more discriminative features. When CKD is applied alone, the accuracy reaches 83.56%. This result is lower than that of FKD, suggesting that category-level distillation alone is less effective for this task. The best result is obtained when FKD and CKD are used together, with an accuracy of 86.25%. This indicates that feature-level and category-level distillation provide complementary gains.

5. Conclusion

This work presents a framework that uses complementary spatial and channel context for plant disease recognition. The core model, a Dual-Channel Feature Enhancement Network, integrates spatial-wise and channel-wise self-attention to emphasize localized lesion cues and discriminative channel semantics. As the teacher model, DC-FEN provides structured knowledge for training a compact student network.

To support deployment on devices with limited resources, we devise a Category-Guided Knowledge Distillation strategy. This strategy transfers both high-level category semantics and fine-grained feature relationships from the teacher to the lightweight student, preserving recognition performance while keeping the deployed student compact. The results indicate that category-level and feature-level distillation can help the student handle visually similar disease patterns.

Overall, this work provides a practical plant disease recognition framework that combines attention-based representation learning with knowledge distillation. Future work will consider multi-modal data, cross-crop generalization, and continual learning settings.