An End-to-End Precision Phenotyping Framework: Rice Panicle Detection and Counting in Complex Fields via Lightweight DETR

1. Introduction

The density per unit area and the spatial distribution pattern of rice panicles are direct agronomic traits determining final crop yield. Accurate, high-throughput quantification of these traits in the field is crucial to overcoming the bottlenecks of traditional yield estimation and breeding screening [1,2]. Recently, unmanned aerial vehicle (UAV) remote sensing has revolutionized agricultural phenotyping [3,4]. By capturing high-resolution field imagery, UAV platforms enable the rapid acquisition of large-scale heading-stage traits.

However, translating raw UAV imagery into actionable agronomic insights faces significant engineering hurdles. Historically, UAV-based phenotyping has relied on a rigid “capture-and-process” paradigm, where heavy image data must be transferred to high-performance ground workstations for offline analysis [5,6]. As precision agriculture evolves towards autonomous, real-time decision-making, deploying high-throughput phenotyping systems directly onto UAV edge devices or autonomous agricultural robots has become imperative to bypass transmission latency and data redundancy [7]. Yet, edge deployment in unstructured agricultural environments introduces profound challenges. First, edge devices possess strictly limited computational power and battery capacities, which prohibits the use of parameter-heavy deep learning models. Second, unlike controlled laboratory conditions, UAVs operating in real fields experience unavoidable altitude fluctuations (e.g., dropping from 20 m to 3 m) due to wind interference and uneven terrain. These altitude shifts cause extreme spatial scale variations of the targets. As recently highlighted in small object detection studies, such multi-scale variations often lead to severe background noise interference and the “semantic collapse” of micro-panicles in standard neural networks [8]. Furthermore, severe leaf occlusion and dynamic lighting reflections from flooded paddies exacerbate the difficulty of automated feature extraction.

To bypass manual feature engineering, the advent of deep learning, particularly Convolutional Neural Networks (CNNs), established a robust automated paradigm. Optimized one-stage detectors have been widely explored for agricultural contexts, such as multi-scale information-enhanced weed detection [9] and crop maturity evaluation combining AI with remote sensing [10]. Initial efforts in automated panicle detection employed two-stage models; for example, Zhang et al. [11] improved Faster R-CNN using multi-scale feature fusion. However, the rapid evolution toward lightweight one-stage architectures has dominated recent literature, particularly for UAV applications. For instance, recent comparative studies have underscored the necessity of evaluating YOLO variants under constrained UAV image datasets for precise yield estimation [12], as well as adapting these models to mitigate severe lighting variations and environmental noise in natural fields [13]. Building on these insights, architectures like EfficientNet-based OE-YOLO [14] and DRPU-YOLO11 [15] have been specifically developed to tackle complex infield backgrounds. This architectural evolution is also continued in recent state-of-the-art models like YOLOv10 [16], which aims for real-time end-to-end detection by eliminating NMS post-processing. To accommodate operational constraints, FRPNet addresses multi-altitude scale variations [17], while LKD-YOLO leverages knowledge distillation to facilitate edge deployment [18]. To further resolve specific unstructured field anomalies, researchers have continuously introduced specialized modules, such as rotated bounding boxes to handle inclined panicles [19], enhanced feature fusion networks to improve counting precision [20], and multi-scale enhancement methods like MFNet [21] to mitigate miss-detections in complex backgrounds. Additionally, the integration of autonomous vision-guided systems directly into field robotics underscores the critical need for robust, real-time detection in complex canopies [22].

Despite these advancements, CNN-based architectures suffer from inherent spatial limitations. Their localized receptive fields restrict the assimilation of global contextual information, making it exceedingly difficult to separate targets from backgrounds during late growth stages when severe panicle adhesion occurs [23,24]. While Transformer architectures circumvent this by establishing long-range pixel dependencies—with DETR [25] transforming object detection into a set prediction problem and RT-DETR [26] achieving real-time efficiency via a hybrid encoder—their specialized application in edge-based agriculture is still in its infancy. Recent breakthroughs have demonstrated the immense potential of hybrid CNN-Transformer architectures in tackling complex agricultural segmentation and detection tasks [27]. However, directly applying generic RT-DETR to UAV imagery exposes a critical flaw: the faint textures of tiny panicles are easily submerged by background noise, and traditional downsampling operations demand excessive computational resources while causing irreversible feature loss for high-altitude targets.

To bridge this fundamental gap between algorithmic precision and edge-deployable engineering, this paper proposes an end-to-end precision phenotyping framework: Panicle-DETR. Tailored specifically for complex field environments and multi-altitude UAV operations, our lightweight framework introduces three core engineered innovations:

• FasterFD (Frequency-Aware CSP Backbone): Inheriting the robust Cross Stage Partial (CSP) hierarchy of YOLOv8, this backbone introduces a radical micro-architectural innovation by upgrading standard bottlenecks into C2f-FasterFD modules. By integrating the Partial Convolution (PConv) paradigm with Frequency Dynamic Convolution (FDConv), it effectively suppresses low-frequency background noise and amplifies faint panicle textures, perfectly optimizing the multi-scale feature extraction process for low-power edge devices.
• LFE (Lossless Feature Encoder): By integrating Space-to-Depth mapping with bidirectional feature pyramids, this encoder achieves complete cross-scale feature reconstruction, successfully overcoming the “semantic collapse” of micro-targets captured from high UAV altitudes.
• Composite Metric Loss: Integrating NWD and Inner-IoU, this formulation compensates for the gradient failure of traditional IoU on overlapping targets, significantly improving bounding box regression and dense counting precision without adding any inference delay.

2. Materials and Methods

2.1. Study Area and Data Preparation

To engineer a highly robust object detection framework capable of adapting to unstructured field environments, accommodating multiple rice cultivars, and extracting scale-invariant features across varying UAV flight altitudes (ranging from 3 m to 20 m), this study constructed a “Composite Multi-Altitude Rice Panicle Dataset”. This composite corpus strategically fuses a massive public benchmark dataset with targeted, ultra-low-altitude self-collected data to simulate a complete UAV operational envelope.

The foundational generalization data for our framework is derived from the DRPD dataset published by Teng et al. [28]. Acquired using an industrial-grade DJI M300 UAV equipped with a Zenmuse P1 imaging sensor, this dataset provides a massive volume of complex field samples. After uniform cropping, the DRPD subset contributes 5,372 high-resolution image patches (512 × 512 pixels), encompassing 229 rice varieties across four critical growth stages (from initial heading to mid-grain filling). Crucially, it contains images captured at three distinct medium-to-high flight altitudes (7 m, 12 m, and 20 m), with Ground Sample Distances (GSD) ranging from approximately 0.08 to 0.25 cm/pixel. This subset provides over 250,000 expert-annotated panicle instances, serving as a solid foundation to guarantee the model’s generalization capability across diverse cultivars and lighting conditions.

However, in practical precision agriculture phenotyping tasks, relying solely on medium-to-high altitude data is insufficient. UAVs frequently need to perform ultra-low-altitude inspections to capture fine-grained morphological traits. To push the model’s multi-scale perception limits to the extreme and validate its adaptability to consumer-grade lightweight UAV sensors, a targeted supplementary data acquisition campaign was conducted in July 2025 at the Gannan Red Seed Industry Rice Research Base (Ganzhou City, Jiangxi Province, China), as illustrated in Figure 1. Unlike the industrial-grade equipment used in DRPD, our collection deliberately utilized a consumer-grade, lightweight UAV (DJI Mini 3) operating strictly at an ultra-low altitude of approximately 3 m. This specific acquisition strategy was designed to capture ultra-high-definition panicle textures and morphological details, acting as a vital spatial-scale supplement to the DRPD dataset.

To standardize spatial dimensions and prepare raw UAV imagery for network ingestion, a systematic data processing pipeline was established, as illustrated in Figure 2. Initially, the raw ultra-low-altitude images acquired by the DJI Mini 3 were processed using a sliding window cropping strategy. This operation partitioned the high-resolution raw images into uniform 512 × 512 pixel patches, ensuring dimensional consistency with the DRPD dataset. Subsequently, a rigorous quality filtering process was applied to discard invalid patches suffering from severe rotor-induced airflow blur or lacking panicle targets.

Following the filtering phase, 646 high-quality ultra-low-altitude image patches were retained. We strictly performed manual annotation on these patches using the LabelImg software. To guarantee cross-dataset compatibility, our annotation protocol adhered exclusively to the bounding box definitions established by the public dataset, ultimately yielding approximately 10,435 high-precision ground-truth bounding boxes for 3 m ultra-low-altitude rice panicles.

Finally, these meticulously processed 3 m data patches were strategically fused with the 7 m, 12 m, and 20 m sub-datasets from DRPD. This fusion yields our final Composite Multi-Altitude Dataset. By integrating these specific altitude gradients, the dataset forces the Panicle-DETR network to learn highly robust, scale-invariant feature representations, successfully simulating the comprehensive altitude fluctuations a UAV experiences during real-world field inspections.

Taking into account the complexity of unstructured field environments, to prevent overfitting and improve the robustness of the model against illumination and viewpoint variations, we implemented multiple online data enhancement strategies during the training phase, as illustrated in Figure 3. Geometric transformations were applied by random horizontal and vertical flipping to simulate various UAV flight headings. To replicate the highly dynamic illumination patterns typical of unstructured field environments—ranging from harsh solar glare to heavy overcast—we introduced stochastic photometric perturbations across the brightness, contrast, and saturation channels. Noise simulation was conducted by adding Gaussian noise to mimic potential sensor interference during UAV operation. These augmentation strategies effectively expanded the feature space of training samples and improved the model’s generalization capability in unseen scenarios.

2.2. Panicle-DETR Network Framework

2.2.1. Overall Network Architecture

To address the profound agronomic engineering challenges of extreme spatial scale variations, high-density occlusion, and the semantic collapse of micro-panicle features in unstructured field environments, this study proposes Panicle-DETR, an improved end-to-end, edge-deployable object detection framework. The overall architecture of Panicle-DETR is illustrated in Figure 4.

To resolve the inherent tension between edge-inference speed and detection precision, we anchored our framework on the RT-DETR architecture. While recent advancements in the YOLO family have successfully achieved end-to-end detection by eliminating NMS post-processing, these models still fundamentally rely on localized spatial receptive fields. This structural constraint makes it exceedingly difficult to separate genuine adjacent targets during the late crop growth stages, where severe panicle adhesion and leaf occlusion occur. RT-DETR circumvents this limitation by employing a Transformer architecture; its self-attention mechanism processes global contextual features, allowing for superior relational modeling among densely packed agricultural targets.

Nevertheless, generic RT-DETR models exhibit a distinct vulnerability in agricultural aerial imagery: they struggle to encode the fine-grained textures of tiny objects, and their default heavy backbones (e.g., ResNet) are too computationally expensive for UAVs. To address this, Panicle-DETR establishes a highly efficient hybrid architecture. We replace the conventional backbone with a customized YOLO-style Cross Stage Partial (CSP) network. Rather than relying solely on standard spatial convolutions, the imagery is routed through this upgraded frequency-domain perception backbone, which explicitly decouples faint panicle signals from background noise while maintaining ultra-high throughput. These refined features are then passed into the LFE encoder, engineered to propagate complete spatial information across multiple scales without the decimation typical of strided downsampling. The entire flow is ultimately resolved by an improved decoder and detection head to deliver robust end-to-end predictions.

2.2.2. FasterFD: Frequency-Aware CSP Feature Extraction Backbone

In complex background images captured by low-altitude UAVs, rice panicles often exhibit extremely fine textures and blurred edges. Consequently, traditional spatial-domain convolutions easily overlook this high-frequency morphological information. From a biomorphological perspective, mature rice panicles possess significant high-frequency texture features defined by drastic grayscale variations, especially compared to flooded soil and mature leaves in the background, which are typically present as low-frequency and smooth continuous signals.

To resolve this while strictly adhering to the stringent inference speed constraints of field-edge devices, we designed the FasterFD backbone. Macro-structurally, FasterFD inherits the robust Cross Stage Partial hierarchy characteristic of the YOLOv8 architecture. It systematically downsamples spatial features via strided convolutions to generate a standard five-level multi-scale representation spanning from P1 to P5. However, generic YOLOv8 architectures rely heavily on standard spatial convolutions within their iconic Cross Stage Partial Bottleneck, or C2f, modules. This over-reliance renders faint panicle features highly susceptible to being submerged by complex environmental noise while simultaneously generating severe computational redundancy.

To break this bottleneck, we introduce a radical micro-architectural innovation. As illustrated in Figure 5, we systematically upgrade standard C2f modules into our proposed C2f-FasterFD modules. This upgraded architecture integrates the highly efficient computational paradigm of FasterNet [29] with advanced frequency-domain filtering, striking an optimal balance between high-frequency perception capabilities and edge-computing efficiency.

The architecture of the C2f-FasterFD module is governed by two core principles. The first involves partial channel processing. Within the bottleneck structure, to explicitly minimize memory access costs on UAV payloads, we adopt a partial convolutional channel splitting strategy. An input feature map $\mathbf{X}\in {\mathbb{R}}^{C\times H\times W}$ is partitioned along the channel dimension into an active subset ${\mathbf{X}}_{\mathrm{active}}\in {\mathbb{R}}^{{\displaystyle \frac{C}{r}}\times H\times W}$, using a standard split ratio of $r=4$, along with an unmodified identity subset ${\mathbf{X}}_{\mathrm{identity}}$.

The second principle focuses on the enhancement of the frequency-domain features. For the active channel subset, we abandon the traditional spatial convolution $3\times 3$ and substitute it with the frequency dynamic convolution [30]. By projecting spatial signals into the frequency domain via a Fast Fourier Transform, this operation inherently functions as a global high-pass filter. This transformation intrinsically broadens the receptive field, actively amplifying the high-frequency granular details characteristic of panicle textures while aggressively attenuating low-frequency background noise. Mathematically, the forward pass for the active channel subset is defined as:

$${\mathbf{X}}_{\mathrm{active}}^{\prime }={\mathcal{F}}^{-1}(\mathbf{K}\odot \mathcal{F}\left({\mathbf{X}}_{\mathrm{active}}\right))$$

(1)

In this formulation, $\mathcal{F}(·)$ and ${\mathcal{F}}^{-1}(·)$ signify the mapping to the frequency domain and its corresponding inverse. The operator ⊙ denotes the Hadamard product executed within the frequency domain, modulated by $\mathbf{K}$, a learnable complex-valued weight matrix. By integrating this frequency-aware projection directly into the YOLO-style hierarchical topology, FasterFD not only retains ultra-high throughput but significantly enhances the network’s resilience against blurred or low-resolution targets in dynamic field environments.

2.2.3. LFE: Lossless Feature Encoder

Resolving the extreme spatial disparities inherent in multi-altitude UAV surveys (varying from 3 m to 20 m) requires a specialized feature fusion architecture. Agricultural micro-target detection is fundamentally constrained by a spatial-semantic dichotomy: early network layers retain the critical textural and geometric granularity of panicles but lack macroscopic context, whereas deep layers encode robust semantics at the cost of severe spatial degradation.

Conventional feature pyramids frequently fail to bridge this divide. Their standard strided downsampling operations (Stride-2 Conv) irreversibly decimate the physical pixel footprints of high-altitude targets—causing severe semantic degradation for micro-panicles spanning merely a handful of pixels.

To address this structural imbalance and maximize spatial information retention, this study proposes the Lossless Feature Encoder (LFE). This module systematically reconstructs the cross-scale feature flow pipeline. First, it integrates the bidirectional topological structure and fast normalized weighted fusion mechanism of BiFPN [31] (Figure 6a). The network adaptively adjusts the fusion weights based on the contribution of the input features, thereby intelligently prioritizing shallow layers rich in micro-target morphology and preventing them from being overshadowed by deep-level macroscopic semantics.

Crucially, to effectively prevent the spatial information loss caused by traditional convolutions during pyramid construction, we innovatively embed SPDConv [32] seamlessly into the cross-scale downsampling paths of the LFE. As shown in Figure 6b, SPDConv leverages a Space-to-Depth mapping paradigm. It rearranges spatial information into the channel dimension through a precise slicing operation, achieving effectively lossless downsampling. Specifically, for an input feature map $\mathbf{X}$ of size $S\times S\times C_{\mathrm{in}}$, it is transformed into an output feature map ${\mathbf{X}}^{\prime }$ of size $S/2\times S/2\times 4C_{\mathrm{in}}$. The mapping relationship is defined as:

$${\mathbf{X}}_{x,y}^{\prime }=[{\mathbf{X}}_{2x,2y},{\mathbf{X}}_{2x+1,2y},{\mathbf{X}}_{2x,2y+1},{\mathbf{X}}_{2x+1,2y+1}]$$

(2)

In summary, the LFE optimally synergizes the adaptive attention of BiFPN with the spatial-preserving characteristics of SPDConv. It ensures that the granular textures of ultra-small panicles captured at 20 m altitudes are robustly transmitted and fused into deep semantic features, significantly elevating the model’s multi-scale robustness under extreme UAV altitude variations.

2.2.4. Adjustment of the Loss Function

In densely planted paddies, severe occlusion and morphological adhesion among tiny panicles expose two fundamental flaws in traditional Intersection over Union (IoU) loss. First, non-overlapping bounding boxes yield a zero IoU, preventing meaningful gradient backpropagation. Second, marginal positional deviations in micro-targets trigger precipitous IoU drops, destabilizing model convergence. To resolve these issues, we designed a composite regression loss strategy combining metric learning and auxiliary bounding boxes.

• Normalized Wasserstein Distance Loss

First, to address the issue that small objects are extremely sensitive to positional deviations, we introduce the NWD loss[33]. Unlike the traditional IoU calculation based on geometric overlap, NWD models the bounding box $B=(c_x,c_y,w,h)$ as a 2D Gaussian distribution $\mathcal{N}(\mu ,\mathrm{\Sigma })$, where the mean vector $\mu$ and the covariance matrix $\mathrm{\Sigma }$ are defined as:

$$\mu =\left[\begin{array}{c}{c}_x\\ c_y\end{array}\right],\mathrm{\Sigma }=\left[\begin{array}{cc}{\displaystyle \frac{w^2}{4}}& 0\\ 0& {\displaystyle \frac{h^2}{4}}\end{array}\right]$$

(3)

By calculating the second-order Wasserstein distance $W_2^2$ between the predicted distribution ${\mathcal{N}}_p$ and the ground truth distribution ${\mathcal{N}}_g$, we construct $L_{\mathrm{NWD}}$:

$$L_{\mathrm{NWD}}=1-exp\left(-{\displaystyle \frac{\sqrt{\left|\right|{\mu }_p-{\mu }_g{\left|\right|}_2^2+\left|\right|{\mathrm{\Sigma }}_p^{1/2}-{\mathrm{\Sigma }}_g^{1/2}{\left|\right|}_F^2}}{C}}\right)$$

(4)

By measuring distributional similarity, NWD smooths the drastic gradient fluctuations caused by minor positional deviations, enabling the model to maintain stable learning capabilities when handling non-overlapping or slightly overlapping samples.

2.

Inner-IoU Loss

While NWD resolves the metric failure for distant targets, convergence speed remains suboptimal during the fine-grained regression stage for highly overlapping samples. To optimize this, we integrated the Inner-IoU loss [34]. Traditional IoU provides uniform gradients irrespective of sample quality. By applying a spatial scale factor $\mathrm{ratio}$ (empirically set to 0.7), Inner-IoU generates contracted “auxiliary inner boxes” within the original anchors:

$$\begin{array}{cc}\hfill b_l^{\mathrm{inner}}& =c_x-{\displaystyle \frac{w·\mathrm{ratio}}{2}},b_r^{\mathrm{inner}}=c_x+{\displaystyle \frac{w·\mathrm{ratio}}{2}}\hfill \\ \hfill b_t^{\mathrm{inner}}& =c_y-{\displaystyle \frac{h·\mathrm{ratio}}{2}},b_b^{\mathrm{inner}}=c_y+{\displaystyle \frac{h·\mathrm{ratio}}{2}}\hfill \end{array}$$

(5)

By exclusively computing the intersection ratio between these auxiliary inner boxes (${\mathrm{IoU}}^{\mathrm{inner}}$), the absolute value of the regression gradient is significantly amplified, thereby rapidly accelerating the bounding box refinement for dense, adhered panicles.

3.

Total Loss Formulation

Ultimately, Panicle-DETR is optimized through a synergistic multi-task objective. The total loss formulation, $L_{\mathrm{Total}}$, integrates Varifocal Loss ($L_{\mathrm{VFL}}$) for classification with the dual-metric bounding box regression mechanism. To mitigate the extreme scale disparities, the regression penalty seamlessly amalgamates the NWD and Inner-IoU metrics via an interpolation coefficient $\alpha$:

$$L_{\mathrm{Total}}={\lambda }_{\mathrm{cls}}{L}_{\mathrm{VFL}}+{\lambda }_{\mathrm{box}}\left(\alpha L_{\mathrm{Inner}-\mathrm{IoU}}+(1-\alpha )L_{\mathrm{NWD}}\right)+{\lambda }_{L1}{L}_{L1}$$

(6)

Here, ${\lambda }_{\mathrm{cls}}$, ${\lambda }_{\mathrm{box}}$, and ${\lambda }_{L1}$ serve as structural hyperparameters to explicitly regulate the gradient contribution of each component. By dynamically coupling the distributional sensitivity of NWD with the accelerated boundary convergence of Inner-IoU, this tripartite optimization scheme guarantees exceptional localization resilience amidst hyper-dense, unstructured agricultural conditions.

3. Results

3.1. Experimental Setup

3.1.1. Experimental Environment

To ensure consistent evaluation and eliminate potential hardware-induced performance bottlenecks during deep learning iterations, all models were trained and tested on a workstation equipped with an Intel Core i7-14700KF processor, 64 GB of RAM, and an NVIDIA GeForce RTX 4080 SUPER GPU (16 GB VRAM). Furthermore, the software environment was strictly configured with Ubuntu 22.04 LTS, Python 3.10, PyTorch 2.5.1, and CUDA 12.4 to maintain a highly standardized and reproducible experimental pipeline.

3.1.2. Experimental Parameters

To objectively evaluate the feature extraction capabilities of the models without bias from pre-existing weights, all networks were trained entirely from scratch. Input images were standardized to a spatial resolution of $640\times 640$ pixels. The detailed configuration of the unified training hyperparameters is summarized in Table 1.

As outlined in Table 1, the AdamW optimizer was utilized in conjunction with a Cosine Annealing scheduler to prevent premature convergence in local minima. A weight decay factor was systematically applied to suppress overfitting on complex background noise, such as water reflections and soil. To ensure optimal convergence while minimizing redundant computation, training was capped at 300 epochs, supplemented by a 50-epoch early stopping mechanism monitored via the validation loss.

3.1.3. Evaluation Metrics

To rigorously quantify the efficacy of Panicle-DETR within unstructured agricultural settings, we established a tripartite evaluation framework encompassing fundamental detection capability, holistic localization accuracy, and agronomic counting variance.

(1) Fundamental Detection Metrics

Recognizing that false negatives (missed panicles) compromise yield projections more severely than false positives (background misclassifications), we prioritized Precision (P), Recall (R), and the F1-Score. Specifically, P quantifies the exactness of the predicted bounding boxes, R evaluates the model’s capacity to retrieve all genuine panicles, and the F1-Score acts as their harmonic mean to indicate overall architectural stability. The corresponding formulations are:

$$P={\displaystyle \frac{TP}{TP+FP}}\times 100\%$$

(7)

$$R={\displaystyle \frac{TP}{TP+FN}}\times 100\%$$

(8)

$$F1={\displaystyle \frac{2\times P\times R}{P+R}}$$

(9)

Here, $TP$ denotes the volume of accurately localized panicles, $FP$ designates background artifacts (such as foliage or soil) incorrectly classified as targets, and $FN$ represents the ground-truth panicles overlooked by the detector.

(2) Comprehensive Accuracy Metrics

In alignment with rigorous COCO evaluation protocols, Mean Average Precision (mAP) was adopted as the primary holistic metric. It aggregates the Average Precision ($AP$) over the entire precision-recall trajectory. The definitions are expressed as:

$$AP={\int }_0^{1}P\left(R\right)dR,mAP={\displaystyle \frac{1}{N}}\sum _{i=1}^{N}A{P}_i$$

(10)

In this context, N signifies the total class count (with $N=1$ for our single-class panicle task). We specifically reported ${\mathrm{mAP}}_{50}$ (at an Intersection over Union threshold of 0.5) to baseline the general detection performance, alongside ${\mathrm{mAP}}_{50-95}$ (averaged across IoU thresholds from 0.5 to 0.95 in 0.05 increments) to rigorously examine the accuracy of bounding box regression under stringent boundary constraints.

(3) Agronomic Counting Metrics

To validate the utility of the model for downstream yield prediction, the precision of panicle counting was assessed by measuring the deviation between the algorithmic predictions ($C_{\mathrm{pred}}$) and the manual ground-truth annotations ($C_{\mathrm{gt}}$). This evaluation utilized the Mean Absolute Error (MAE), the Root Mean Square Error (RMSE), and the Coefficient of Determination ($R^2$):

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n|C_{\mathrm{pred},i}-C_{\mathrm{gt},i}|$$

(11)

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(C_{\mathrm{pred},i}-C_{\mathrm{gt},i})}^2}$$

(12)

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{(C_{\mathrm{gt},i}-C_{\mathrm{pred},i})}^2}{{\sum }_{i=1}^n{(C_{\mathrm{gt},i}-{\overline{C}}_{\mathrm{gt}})}^2}}$$

(13)

where n indicates the total volume of test set images, and ${\overline{C}}_{\mathrm{gt}}$ is the empirical mean of the actual panicle distributions. Minimized MAE and RMSE scores, coupled with an $R^2$ approaching unity, denote exceptional counting consistency, which is an absolute prerequisite for high-throughput phenotyping and large-scale digital agriculture.

3.2. Ablation Study of Engineered Innovations

To quantify the independent contributions and synergistic effects of the proposed architectural modifications, we conducted a combinatorial ablation study. The evaluated components include the FasterFD backbone, the Bidirectional Feature Pyramid Network (BiFPN), and the composite bounding box regression loss (NWD + Inner-IoU). As detailed in Table 2, the baseline model (RT-DETR-r18) was systematically augmented with individual modules and their pairwise combinations to isolate their impacts on terminal localization accuracy and computational overhead.

These architectural permutations elucidate several mechanisms explicitly tailored for UAV constraints. Given that inference economy is paramount for battery-powered drones, integrating the FasterFD module substantially reduced computational demand. Specifically, Model 1 decreased GFLOPs by 26.5% (from 56.9 to 41.8) compared to the baseline, while concurrently improving mAP@50 to 92.95%. This demonstrates that projecting feature extraction into the frequency domain effectively filters out redundant, low-frequency background noise (e.g., homogeneous soil or water reflections), redirecting constrained computational bandwidth exclusively toward the high-frequency textural details of panicles.

Building upon this efficient foundation, the LFE module addresses the spatial degradation of micro-targets. While isolating LFE (Model 2) increased mAP@50 to 93.20% and improved Recall, its dense bidirectional fusion inherently escalated the computational load to 64.7 GFLOPs. This uplift confirms the efficacy of the Space-to-Depth mapping mechanism, which circumvents physical pixel decimation to preserve the minute footprints of high-altitude panicles. Crucially, when LFE is coupled with FasterFD (Model 4), the computational savings from the spectral backbone fully absorb the encoder’s overhead. This synergy stabilizes the final GFLOPs at 53.0 while pushing mAP@50 to 93.60%, achieving a highly competitive balance between multi-altitude accuracy and edge efficiency.

Finally, the composite loss function (NWD + Inner-IoU) optimizes boundaries for dense clusters with zero additional inference overhead. Applied in isolation (Model 3), the gain remained marginal (92.55% mAP@50) because traditional convolutions fail to extract sufficiently clear micro-target features, rendering the advanced metric loss ineffective on highly blurred boxes. However, a powerful synergistic effect emerges in the complete architecture. Once FasterFD and LFE establish high-fidelity, scale-invariant representations, the composite loss decisively pushes Precision to 90.97% and mAP@50 to 93.94%. This verifies that prioritizing distributional similarity over geometric overlap effectively decouples adherent panicles, provided the preceding network supplies robust structural features.

3.2.1. Comparative Analysis of Backbone Architectures

To further validate the selection of FasterFD, we benchmarked it against various mainstream vision backbones. All candidates were integrated into the identical Panicle-DETR paradigm (equipped with BiFPN and composite loss) to strictly isolate their feature extraction capabilities. These encompassed parameter-heavy architectures (ResNet-50 [35], HGNetv2-L [26]), lightweight networks (MobileNetV3 [36], StarNet [37]), and state-of-the-art efficient models (FasterNet [29], EfficientNetV2 [38]). As Table 3 demonstrates, FasterFD achieves an optimal trade-off between detection accuracy and edge-inference efficiency.

Unlike traditional heavy networks such as ResNet-50—which guarantee robust accuracy (93.70% mAP@50) but incur a massive computational cost of 108.5 GFLOPs—FasterFD slightly surpasses this precision (93.94%) while reducing the computational load by more than half (53.0 GFLOPs). This effectively eliminates the parameter redundancy that typically restricts real-time UAV deployment. Conversely, while lightweight networks like StarNet and MobileNetV3 maintain compact footprints (40.0 and 47.7 GFLOPs, respectively), their constrained capacities prove insufficient against complex field backgrounds, resulting in severe accuracy degradation (dropping to 91.80% and 90.50% mAP@50).

Most notably, when benchmarked against the original FasterNet architecture (which relies purely on spatial-domain partial convolutions at an economical 41.5 GFLOPs), FasterFD yields a decisive 1.09% enhancement in mAP@50. While vanilla FasterNet exhibits a low footprint, its spatial-only perception struggles with the severe morphological blurring and specular reflections inherent to irrigated paddies. By embedding partial convolution principles into the robust multi-scale CSP hierarchy of YOLOv8 and projecting feature extraction into the frequency domain, our C2f-FasterFD modules trade a moderate increase in computation for a significant breakthrough in localization accuracy. This confirms that for agricultural targets with blurred edges and severe environmental noise, coupling a CSP topology with frequency-domain perception provides a far more resilient foundation than purely spatial architectures.

3.3. Comparative Experiments on the Composite Dataset

To comprehensively validate the superiority of the Panicle-DETR framework in unstructured agricultural environments, we conducted extensive comparative experiments against mainstream state-of-the-art architectures. The evaluation integrates fundamental detection metrics, agronomic counting stability, and qualitative visual validation.

3.3.1. Quantitative Detection Performance and Training Dynamics

Detailed object detection performance and complexity metrics for each model on the test set are presented in Table 4. The quantitative evaluation indicates that Panicle-DETR achieves a highly effective equilibrium between detection accuracy and computational efficiency, specifically tailored for edge deployment.

For fundamental localization accuracy, an outstanding Precision of 90.97% and an F1-Score of 89.63% were recorded for Panicle-DETR, establishing a distinct superiority over all evaluated baselines. When evaluated under the standard mAP@50 criterion, a peak score of 93.94% was attained by our architecture. It is noteworthy that the highly optimized YOLOv8m [41] and the latest YOLOv12m [43] models also demonstrated exceptional capabilities, achieving mAP@50 scores of 93.65% and 93.80%, respectively. However, Panicle-DETR consistently maintains a clear performance margin over these parameter-heavy networks, all while requiring significantly fewer parameters (13.78 M) and computational GFLOPs (53.0).

The performance delta becomes particularly pronounced under stringent spatial overlap thresholds. Specifically, mAP@75 and mAP@50-95 scores of 74.15% and 64.75% were achieved, respectively. This enhanced bounding box regression is directly driven by the integration of the combined NWD and Inner-IoU loss formulation, which enables the predicted bounding boxes to closely encapsulate irregular panicle boundaries.

Beyond terminal accuracy, the optimization trajectory of the proposed network was characterized by rapid convergence and minimal volatility. The comparative training dynamics—mapped across Precision, Recall, and mAP@50 in Figure 7—reveal that while initial learning phases were universally rapid, late-stage optimization in the YOLO lineage and Faster R-CNN was marred by pronounced metric oscillations.

Conversely, the training progression of Panicle-DETR remained inherently stable (Figure 7). This sustained stability is largely facilitated by the frequency-domain backbone (FasterFD) and the BiFPN multi-scale fusion mechanism, which collaboratively streamline the feature extraction process. To guard against overfitting and preserve generalization on unseen agricultural imagery, an early stopping protocol was enforced, automatically terminating the training phase for Panicle-DETR at epoch 236. By curtailing redundant computational cycles, this mechanism underscores the overarching operational efficiency of the architectural design.

3.3.2. Agronomic Counting Efficacy and Stability

In high-throughput phenotypic analysis, detection confidence alone may not fully reflect the reliability of yield prediction. Therefore, a comprehensive quantitative analysis of panicle counting errors was conducted to evaluate model consistency.

Panicle-DETR demonstrated notable improvements in counting consistency, achieving a Mean Absolute Error of 4.28 and a Root Mean Square Error of 8.57. Notably, while YOLOv8m [41] and the recent YOLOv12m [43] exhibit strong object detection capabilities, their respective Mean Absolute Errors (5.05 and 4.85) in the counting task remain comparatively higher. This performance gap suggests that the adaptive multi-scale fusion mechanism of the BiFPN structure helps preserve the physical boundaries of micro-targets among dense and adherent panicles, effectively mitigating the occurrence of false merging and missed detections.

Table 5. Evaluation of panicle counting performance across different models.

Model	MAE ↓	RMSE ↓	${\mathit{R}}^2$↑
Faster R-CNN [39]	8.20	12.50	0.850
YOLOv5m [40]	6.35	10.80	0.895
YOLOv8m [41]	5.05	9.35	0.938
YOLOv10m [16]	5.60	9.95	0.920
YOLOv11m [42]	4.95	9.25	0.942
YOLOv12m [43]	4.85	9.12	0.945
RT-DETR-r18 [26]	6.15	10.20	0.910
Panicle-DETR (Ours)	4.28	8.57	0.957

Table 5. Evaluation of panicle counting performance across different models.

Model	MAE ↓	RMSE ↓	${\mathit{R}}^2$↑
Faster R-CNN [39]	8.20	12.50	0.850
YOLOv5m [40]	6.35	10.80	0.895
YOLOv8m [41]	5.05	9.35	0.938
YOLOv10m [16]	5.60	9.95	0.920
YOLOv11m [42]	4.95	9.25	0.942
YOLOv12m [43]	4.85	9.12	0.945
RT-DETR-r18 [26]	6.15	10.20	0.910
Panicle-DETR (Ours)	4.28	8.57	0.957

To intuitively analyze the response differences of various models when faced with complex field density distributions, linear regression scatter plots of predicted counts versus ground truth counts were evaluated. In actual dense counting tasks in farmlands, significant heteroscedasticity is typically observed; as the true number of targets increases, the prediction error amplifies accordingly. This pattern is clearly visually corroborated in Figure 8. For legacy models such as Faster R-CNN [39] and YOLOv5m [40], when the actual number of rice panicles exceeds 60, the test sample points exhibit a distinct trumpet-shaped divergence, severely deviating from the ideal theoretical fit. Conversely, the prediction scatter points for Panicle-DETR converge tightly around the regression line even in ultra-high-density regions containing over 100 panicles. Achieving a coefficient of determination ($R^2$) of 0.957, our model effectively overcomes the instability caused by heteroscedasticity without systematic bias, demonstrating immense potential for direct application in large-scale field yield estimation.

3.3.3. Qualitative Visual Validation Across Altitudes

To intuitively corroborate the quantitative metrics and demonstrate the operational robustness of Panicle-DETR under real-world agricultural conditions, a comprehensive qualitative visual assessment was conducted across varying UAV flight altitudes. Figure 9 compares the detection outputs of representative baseline models against our proposed architecture at altitudes of 12 m, 7 m, and 3 m, arranged sequentially from left to right.

Visual comparisons confirm that flight altitude significantly dictates the morphological complexity of the panicles. At the 12 m altitude, severe spatial downsampling causes micro-targets to blend into the complex field background. Traditional architectures exhibit notable False Negatives in these regions, failing to capture severely degraded targets, which ultimately skews large-scale yield estimation. In contrast, Panicle-DETR successfully retrieves these minute semantic footprints. This direct visual evidence validates the efficacy of the Lossless Feature Encoder in preserving cross-scale spatial resolution, explicitly ensuring no panicle is lost to high-altitude pixel decimation.

At the optimal 7 m altitude, where panicle density and morphological adhesion become visually overwhelming, baseline models relying on the standard Intersection over Union metric frequently suffer from bounding box merging, manifesting as grouped False Negatives. Driven by the NWD and Inner-IoU composite loss, which shifts the optimization focus from rigid geometric overlap to internal distributional similarity, Panicle-DETR accurately delineates individual boundaries within these highly adhered clusters. Finally, at the 3 m ultra-low altitude, the narrow field of view inherently produces truncated targets at image borders alongside panicles lacking conspicuous visual features. While traditional baseline models struggle with these partial semantics and produce noticeable missed detections, Panicle-DETR maintains exceptional feature extraction robustness. Facilitated by the frequency-aware FasterFD backbone, our model successfully captures subtle morphological cues, ensuring accurate bounding box localization on severely truncated targets that other architectures persistently fail to detect.

3.4. Error Analysis and Model Limitations

A critical appraisal of failure modes is essential to delineate the engineering boundaries of Panicle-DETR in unstructured agricultural environments. A targeted error analysis on the test set isolated specific environmental scenarios and morphological edge cases that compromise model reliability, with representative instances illustrated in Figure 10.

One primary failure mode, depicted in panel (a), involves complex environmental reflections on the water surface within irrigated paddies. Specular reflections of the sky or dense foliage can mimic the semantic granularity and structural characteristics of rice panicles under specific solar angles, occasionally triggering false positives. Concurrently, extreme physical occlusion remains a persistent bottleneck. Although the Lossless Feature Encoder optimizes multi-scale fusion, the defining textural and geometric footprints of a panicle are irreparably suppressed when almost entirely masked by broad flag leaves, inevitably leading to false negatives.

Furthermore, strong illumination saturation introduces notable detection challenges, as observed in panel (b). Under peak solar intensity, extreme brightness washes out the high-frequency textural distinctiveness relied upon by the FasterFD backbone. The attenuated contrast between the target and the background causes spectral features to merge. This phenomenon leads to coupled failures: missed detections arising from semantic loss, and false positives where overexposed background elements, such as withered leaves, are erroneously assigned high objectness scores. These limitations underscore the inherent constraints of purely visual-based detectors in open fields, pointing toward future research in multi-sensor fusion or illumination-invariant data augmentation.

4. Discussion

The deployment of deep learning models for high-throughput phenotyping via Unmanned Aerial Vehicles remains highly constrained by the unstructured nature of open-field agricultural environments. Traditional computer vision architectures, primarily optimized for standardized datasets, frequently exhibit severe performance degradation when confronted with the dynamic challenges of agricultural inspections—most notably, extreme multi-altitude spatial variations, profound computational hardware limits, and severe morphological occlusion. The empirical results demonstrated by Panicle-DETR provide a comprehensive engineering solution to these inherent bottlenecks.

4.1. Overcoming the Spatial-Semantic Dichotomy in Multi-Altitude Phenotyping

One of the primary catalysts for the architectural redesign in this study was the catastrophic semantic collapse observed in conventional models during UAV altitude fluctuations ranging from 3 m to 20 m. As noted in recent 2025 evaluations of small object detection in aerial imagery [44], standard strided convolutions irreversibly decimate the physical pixel footprints of high-altitude micro-targets. While architectures like YOLOv12m achieve exceptional accuracy at a fixed 7 m altitude, their error rates surge unacceptably at higher elevations.

Our proposed Lossless Feature Encoder directly mitigates this vulnerability. By embedding the Space-to-Depth mapping mechanism within the BiFPN topology, Panicle-DETR circumvents pixel decimation, ensuring that the granular textures of micro-panicles—even at severe high-altitude perspectives between 12 m and 20 m—are losslessly preserved and adaptively fused into deep semantic layers. This specific structural innovation explains why Panicle-DETR sustained robust localization accuracy under extreme spatial downsampling, proving that multi-scale robustness in precision agriculture requires explicit spatial preservation and feature distillation strategies, especially around boundary regions of panicles [45].

4.2. Bridging the Gap Between Computer Vision and Agronomic Yield Estimation

In the context of digital agriculture, the ultimate objective of panicle detection is not merely drawing bounding boxes, but delivering highly reliable counting metrics for downstream yield estimation. Many contemporary agricultural vision studies rely exclusively on Mean Average Precision as the terminal metric, which often masks underlying counting deficiencies when assessing complex phenotypic traits such as panicle number per unit area [46].

As previously visualized in our comparative scatter plots, a distinct trumpet-shaped divergence emerged in standard models when the true panicle count exceeded 60 per image. This discrepancy is primarily caused by standard Intersection over Union metrics failing to distinguish heavily adhered panicles, leading to aggressive missed detections during Non-Maximum Suppression. By shifting the regression paradigm from rigid geometric overlap to distributional similarity and amplifying the inner-boundary gradients, Panicle-DETR effectively decoupled these adhered clusters. The resulting reduction in counting error confirms that specifically tailoring the loss function to the morphological characteristics of dense crops is an absolute prerequisite for transitioning from theoretical visual detection to practical intelligent agriculture [47].

4.3. Engineering Viability and Edge-Deployment Trade-offs

For battery-powered agricultural UAVs, computational efficiency dictates the feasibility of real-time onboard processing. Traditional heavy networks such as ResNet-50 demand over 100 GFLOPs, rendering them largely impractical for real-time edge deployment. Conversely, naive lightweight networks suffer severe accuracy drops due to their inability to filter out complex environmental noise, such as water reflections in paddies.

The integration of the FasterFD backbone represents a pivotal engineering trade-off. By explicitly decoupling low-frequency background noise from high-frequency panicle textures within the frequency domain, the network minimizes redundant spatial convolutions. This mechanism allows Panicle-DETR to operate at a highly economical 53.0 GFLOPs while utilizing only 13.78 M parameters, yet achieving an outstanding 93.94% average mAP@50. This establishes a new paradigm for edge-deployable agricultural models: leveraging spectral filtering to maximize the utility of constrained computational bandwidth.models: leveraging spectral filtering to maximize the utility of constrained computational bandwidth.

4.4. Limitations and Future Work

Despite the robust performance of Panicle-DETR, several limitations warrant further investigation. Firstly, the current dataset primarily captures images under adequate daytime illumination. The model’s feature extraction resilience under sub-optimal lighting conditions requires subsequent validation. Secondly, while the parameter count has been strictly minimized, the actual deployment on commercial UAVs necessitates further hardware-specific optimizations. Future research will focus on applying INT8 post-training quantization and TensorRT acceleration to Panicle-DETR, aiming to achieve ultra-low-latency processing directly on edge computing modules without sacrificing the multi-altitude counting accuracy. This strategy explicitly aligns with the latest proven trends of deploying lightweight AI models on agricultural edge devices for real-time inference [48].

5. Conclusions

The transition of high-throughput crop phenotyping to autonomous UAVs is primarily hindered by the computational limits of edge devices and the extreme spatial variability of open-field environments. To resolve this spatial-semantic dichotomy, this study introduces Panicle-DETR, an architecture that reconstructs the feature extraction paradigm without relying on brute-force parameter scaling. By projecting visual perception into the frequency domain via the FasterFD backbone, the network explicitly decouples complex environmental noise from critical high-frequency agronomic textures. By integrating the Lossless Feature Encoder with a composite regression metric based on NWD and Inner-IoU, the framework eliminates the pixel decimation inherent to traditional strided convolutions, effectively resolving the morphological adhesion of dense panicle clusters under diverse altitude-induced spatial variations.

Empirically, these architectural innovations establish a highly competitive state-of-the-art for agricultural edge-inference. Panicle-DETR achieves a peak mAP@50 of 93.94% and translates theoretical detection into reliable agronomic metrics, driving the counting Mean Absolute Error down to an exceptional 4.28. Crucially, this robust multi-altitude performance is sustained at an economical computational footprint of 53.0 GFLOPs and 13.78 M parameters. Ultimately, Panicle-DETR transcends conventional bounding-box detection to establish a scalable, hardware-efficient foundation for real-time, onboard yield estimation, paving the way for uninterrupted digital crop management without the prerequisite of cloud computing.