Open-Set UAV Signal Identification Using Learnable Embeddings and Energy-Based Inference

1. Introduction

UAVs are now widely used across many fields, including smart agriculture [1], re-mote sensing [2], emergency response [3], logistics [4], and infrastructure inspection [5]. However, the rapid increase in low-altitude UAV operations has brought new challenges for airspace safety. Unauthorized or unknown drones can interfere with navigation and communication systems and even pose threats to critical infrastructure and national defense [6,7]. Therefore, reliable identification of UAV communication signals and detection of unknown or suspicious UAVs is essential for maintaining situational awareness in complex electromagnetic environments.

At present, UAV identification is mainly based on acoustic, visual, radar, and radio-frequency (RF) signals. Early works mainly relied on hand-crafted features and traditional machine learning models. For example, Muhammad et al. [8] used Mel-frequency cepstral coefficients from UAV sounds and trained support vector machines (SVMs) for recognition. Chu et al. [9] applied histograms of oriented gradients for visual detection, and Zhang et al. [10] used short-time Fourier transform and principal component analysis for radar-based classification. Nie et al. [11] combined fractal dimension and dual-spectrum features to extract RF fingerprints for UAV recognition. Although these methods can work in simple cases, their reliance on static, manual features makes it hard to adapt to noisy or changing environments.

Among all these data sources, RF signals are particularly valuable because they capture the communication behavior and operational status of UAVs. This makes them suitable for continuous monitoring and anti-UAV applications. In recent years, deep learning has greatly improved UAV signal recognition, since neural networks can learn hierarchical and discriminative features directly from raw IQ data. In other research fields, studies have explored many network designs [12,13,14,15] and learning strategies, such as few-shot [16,17], semi-supervised [18,19], self-supervised [20,21], and transfer learning [22,23], to improve model performance in difficult conditions. Inspired by these, Akter et al. [24] used convolutional neural networks (CNNs) with angle-of-arrival features for UAV recognition; Domenico et al. [25] proposed a real-time RF identification framework; and Cai et al. [26] built a lightweight multi-scale CNN for efficient UAV fingerprinting.

However, most existing works still assume a closed-set condition, in which all test classes are seen during training. In real UAV communication, this assumption is not true. New modulation types and private protocols often appear. Traditional Softmax-based models [27] must classify each input into a single known class and cannot recognize or reject new signals. To solve this problem, open-set recognition (OSR) has been proposed. OSR aims to build a model that not only classifies known samples correctly but also detects and rejects unknown samples.

Figure 1 shows the basic concept of OSR. In a closed-set scenario, the model performs well because all test samples come from known classes. However, when unknown samples appear, the closed-set model usually misclassifies them as one of the known classes. In contrast, an open-set model can not only correctly recognize known samples but also reject unknown ones. The concept of OSR was first introduced by Scheirer et al. [28]. Early studies used traditional classifiers like SVM [29], sparse representation [30], and k-nearest neighbors [31]. Later, Bendale et al. [32] proposed OpenMax, which brought OSR into the deep learning era. After that, researchers extended OSR by using counterfactual samples [33], generative adversarial networks [34,35], and reciprocal point learning [36,37]. Geng et al. [38] developed an OSR method based on a hierarchical Dirichlet process, and Wang et al. [39] employed energy modeling to construct high-energy regions for unknown detection.

Generally, OSR methods can be divided into two types: generative and discriminative. Generative methods try to create fake unknown samples, but it is hard to make them realistic. Discriminative methods aim to construct compact, separable feature spaces. However, they still face some problems, such as limited geometric diversity and sensitivity to noise or channel drift. Many of them use a single latent space, which makes it difficult to balance separability and stability. Also, fixed rejection thresholds reduce the ability to adapt to changing RF conditions. Although OSR has been applied to other fields such as bias detection [40] and pathogen identification [41], it is still rarely used for UAV RF signals. This task is difficult because UAVs often share similar frequency bands, have small differences between classes, and show strong non-stationary motion features, which make detecting unseen signals even harder.

To deal with these challenges, we propose the GE-OSR method for UAV signal classification. Unlike previous works, GE-OSR combines geometric embedding with energy-based modeling into a single framework. This design helps the model learn both the feature structure and the energy distribution, thereby achieving better recognition of known samples and stronger rejection of unknown ones. The main contributions of this paper are as follows:

• A time-frequency convolutional hybrid network for UAV signal representation. Considering the complex and variable patterns of UAV communication signals, a time-frequency convolutional hybrid network is introduced, combining time-frequency analysis with convolutional feature extraction, and ultimately providing more stable and representative signal features.
• A geometric embedding mechanism that enhances intra-class compactness and inter-class separability. For a clearer and more structured feature space, a geometric embedding mechanism is developed, incorporating learnable class embeddings and margin-aware constraints, which eventually lead to compact intra-class clusters and larger distances between classes.
• An energy-based regularization strategy for learning discriminative energy distributions. Faced with the difficulty of distinguishing known and unknown samples in open-set scenarios, an energy-based regularization strategy is adopted, consisting of an explicit energy formulation and its regularization term, and ultimately forming a more discriminative energy landscape.
• An adaptive energy threshold for open-set rejection. Instead of relying on a fixed threshold, an adaptive energy thresholding mechanism is introduced, using the empirical energy distribution of known classes, and finally achieving more reliable rejection of unknown signals under varying openness levels.

2. Materials and Methods

2.1. Feature Extractor

2.1.1. Time-Frequency Convolutional Hybrid Network

UAV RF signals are usually represented as in-phase and quadrature (IQ) sequences, which encode both temporal variations and spectral signatures. These signals evolve as they distribute energy across multiple frequency components, forming a complex pattern spanning two correlated domains. Conventional feature extractors often process them in isolation—either focusing on the time waveform or on its frequency spectrum. This separation makes it difficult for the model to learn how temporal and spectral cues interact. As a result, such approaches tend to miss critical cross-domain dependencies, limiting their capacity to capture the whole structure of UAV signal dynamics and to build strongly discriminative representations.

To overcome this limitation, we designed a time-frequency convolutional hybrid network. This structure leverages convolutional layers for local pattern modeling and Transformer self-attention for global dependency learning, enabling the network to build multi-scale representations that remain stable even under heavy noise. The overall architecture, shown in Figure 2, comprises two key modules: the time-frequency feature merging (TFFM) module and the convolutional-Transformer hybrid block (CTBlock).

2.1.2. Time-Frequency Feature Merging Module

The temporal and spectral components of UAV IQ signals differ substantially in scale, statistical distributions, and noise sensitivity. Simply concatenating them tends to create an imbalance between modalities and weakens the fusion effect. The proposed TFFM module addresses this by using a dual-branch structure that separately processes the time-domain and frequency-domain information, and then fuses them through a learnable attention mechanism.

As shown in Figure 3, the time branch applies a 1D convolution on the raw IQ sequence to capture short-term temporal features. The frequency branch first computes the fast Fourier transform (FFT) of the IQ signals and takes only the first half of the spectrum to avoid redundancy from the conjugate symmetry of real-valued signals. The spectral magnitudes are compressed using a logarithmic transformation $log\left(\right|FFT|+1)$ to stabilize the dynamic range, and then passed through a 1D convolution to extract frequency features.

The outputs of the two branches have different lengths and are aligned by upsampling the frequency features to match the length of the time features. Each branch then passes through a 1 × 1 convolution with a residual connection to maintain feature consistency and channel compatibility. Finally, a lightweight convolution-based attention module computes time-wise fusion weights for the two branches. The attention weights are normalized with Softmax and applied to each branch to produce a fused representation. The final feature is a weighted combination of time and frequency features at each time step.

2.1.3. Convolutional-Transformer Hybrid Block

After fusing time and frequency information, the feature sequence still contains both local patterns and long-range dependencies. Convolutional layers are good at capturing local structures, while Transformer encoders are better at modeling global relationships. To leverage both, the CTBlock combines them in parallel, enabling the network to learn multi-scale features simultaneously.

As shown in Figure 4, the convolutional path uses large-kernel depthwise convolutions followed by pointwise projection to efficiently extract localized spatial-temporal features. Channel expansion is applied before pointwise projection to enrich local representations. In the Transformer path, the input sequence is first transposed and passed through Transformer encoder layers, where the self-attention mechanism captures long-range dependencies. The output is transposed back then. A learnable scalar gate with Sigmoid activation adaptively balances the contributions of the two paths. The fused output is finally added to the original input via a residual connection.

2.2. GE-OSR: An Open-Set Recognition Framework Based on Geometry-Energy Joint Modeling

2.2.1. Framework Overview

The GE-OSR framework is designed to utilize both the geometric relationships of feature vectors and their energy distributions. Its main goal is to correctly classify known UAV signal classes while rejecting unknown or unfamiliar signals. The overall architecture is shown in Figure 5.

As illustrated, GE-OSR contains four core components. First, a convolution-Transformer feature extractor fuses time and frequency information, generating multi-scale and discriminative representations of UAV signals. Second, a set of learnable class embeddings is introduced, which are optimized using a dual-constraint embedding loss (DCEL). This loss encourages intra-class compactness and enlarges inter-class separation, forming a feature space with clear geometric organization. Third, a free energy alignment loss (FEAL) shapes low-energy regions for known classes and high-energy regions for unknowns, providing a reliable energy-based boundary. Finally, during inference, an adaptive energy threshold is applied to distinguish unknown signals from known ones, enabling both closed-set classification and open-set rejection within a unified framework.

2.2.2. Feature Metrics and Class Embedding Initialization

The feature vectors produced by the extractor, denoted as ${\mathit{z}}_i$, are normalized to unit length so that differences among samples are reflected only in their directions. Cosine similarity is adopted to measure the relationships between features and class embeddings, because it is bounded between -1 and 1 and provides a stable geometric interpretation when defining category boundaries. Given two feature vectors $\mathit{x}$ and $\mathit{y}$, their cosine similarity is defined as:

$$sim\left(\mathit{x},\mathit{y}\right)={\displaystyle \frac{\mathit{x}\cdot \mathit{y}}{\left|\right|\mathit{x}\left|\right|\cdot \left|\right|\mathit{y}\left|\right|}}$$

(1)

In UAV signal feature space, class distributions are often uneven, and boundaries between nearby categories may become ambiguous, especially under noisy or overlapping conditions. To alleviate this, GE-OSR introduces a set of learnable class embeddings:

$$\mathit{C}=\left\{{\mathit{c}}_1,{\mathit{c}}_2,\dots ,{\mathit{c}}_K\right\}$$

(2)

where $K$ represents the number of known classes, ${\mathit{c}}_k\in {\mathbb{R}}^d$, and $d$ is the feature dimension of each sample ${\mathit{z}}_i$. During initialization, every class embedding ${\mathit{c}}_j$ is sampled independently from a Gaussian distribution and then normalized onto the unit hypersphere:

$${\tilde{\mathit{c}}}_{\mathit{j}}~\mathit{N}\left(0,{\mathit{I}}_{\mathit{d}}\right),{\mathit{c}}_{\mathit{j}}={\displaystyle \frac{{\tilde{\mathit{c}}}_{\mathit{j}}}{{\left|\right|{\tilde{\mathit{c}}}_{\mathit{j}}\left|\right|}_2}}$$

(3)

This process distributes the embeddings roughly uniformly across the hypersphere, ensuring that they are nearly orthogonal to one another. Such spatial diversity serves as a substantial geometric prior, preventing early-stage collapse of class representations. As training progresses, these embeddings are treated as learnable parameters that gradually adapt through backpropagation, aligning with the empirical feature distributions of the known UAV signal classes.

2.2.3. Dual-Constraint Embedding Loss

To encourage both intra-class compactness and inter-class separability, the DCEL applies explicit geometric regularization in the embedding space. Let ${\mathit{z}}_i\in {\mathbb{R}}^d$ denote the feature vector of the $i$-th sample, $y_i$ its label, and ${\mathit{c}}_k\in {\mathbb{R}}^d$ the embedding vector corresponding to class $k$. The intra-class term encourages features to remain close to their corresponding class embeddings, thereby improving cohesion among samples of the same category. It is defined as:

$$L_{intra}=\mathit{log}\left(1+\sum _{i:y_i=k}{e}^{(-\alpha (\mathit{sim}\left({\mathit{z}}_i,{\mathit{c}}_k\right)-\delta \left)\right)}\right)$$

(4)

where $\alpha$ controls the sensitivity to deviation in similarity, and Delta specifies a soft-margin threshold between categories. By minimizing $L_{intra}$, the model continuously pulls features toward their corresponding embeddings, leading to dense and stable clusters within the representation space.

To avoid cluster overlap, the DCEL incorporates an inter-class separation term, which penalizes samples that appear excessively similar to incorrect class embeddings:

$$L_{inter}=\mathit{log}\left(1+\sum _{i:y_i\ne k}{e}^{\alpha (\mathit{sim}\left({\mathit{z}}_i,{\mathit{c}}_k\right)+\delta )}\right)$$

(5)

When a feature exhibits high similarity to an unrelated class embedding, $L_{inter}$ rises sharply, forcing the network to expand the margins between classes.

The total loss is then defined as:

$$L_{DCE}=L_{intra}+L_{inter}$$

(6)

Together, these two terms create a structured embedding space in which features within the same category are tightly clustered, while features from different classes remain clearly separated.

2.2.4. Free Energy Alignment Loss

While $L_{DCE}$ shapes a discriminative feature geometry, it does not by itself provide a direct mechanism for identifying unknown inputs. To make this possible, we introduce the FEAL, inspired by the energy-based out-of-distribution detection work of Liu et al. [42], and adapt it to the geometry of our learned embeddings. The underlying idea is simple but effective: samples belonging to known categories should lie in low-energy regions. In contrast, those from unknown or unseen categories should lie in the high-energy areas. By explicitly minimizing the energy of known samples, the network learns to create compact low-energy basins in the representation space. Unlike earlier studies that convert Softmax logits into energy, we define energy directly through the cosine distance between a feature vector and the learnable class embeddings—thus maintaining consistency with the geometric modeling in the DCEL.

The cosine distance between two vectors $\mathit{x}$ and $\mathit{y}$ is defined as:

$$D(\mathit{x},\mathit{y})=1-sim(\mathit{x},\mathit{y})$$

(7)

For a given feature $\mathit{z}$, its distances to all class embeddings form the set:

$$\mathit{D}\left(\mathit{z},\mathit{C}\right)=\{D\left(\mathit{z},{\mathit{c}}_1\right),\dots ,D\left(\mathit{z},{\mathit{c}}_K\right)\}$$

(8)

and are mapped to the free energy as follows:

$$E\left(\mathit{z}\right)=-{\displaystyle \frac{1}{T}}log\left(\sum _{k=1}^K{e}^{-TD\left(\mathit{z},{\mathit{c}}_k\right)}\right)$$

(9)

where $T$ is a temperature parameter controlling the smoothness of the energy landscape.

Intuitively, a feature vector close to a class embedding yields low energy, whereas one distant from all embeddings results in high energy. This property makes $E\left(\mathit{z}\right)$ a natural criterion for distinguishing known from unknown signals.

To explicitly constrain known features toward the desired low-energy regions, the FEAL is defined as:

$$L_{FEA}={\displaystyle \frac{1}{N}}\sum _{i=1}^N(E\left({\mathit{z}}_i\right)-E_0{)}^2$$

(10)

where $E_0$ is the target energy level and $N$ is the total number of samples. Minimizing $L_{FEA}$ encourages the free energy of known samples to converge around $E_0$, forming compact, stable low-energy zones within the feature space.

Finally, the overall training objective combines the geometric and energy-based losses:

$$L={{\lambda }_{1}L}_{DCE}+{\lambda }_2{L}_{FEA}$$

(11)

where ${\lambda }_1$ and ${\lambda }_2$ balance the contributions of the two terms. Joint optimization of DCEL and FEAL enables the model to construct a tightly clustered, energetically stable feature space.

2.2.5. Adaptive Energy Threshold Discrimination

In dynamic RF environments, the statistical distribution of signal features may shift during training or testing. A fixed energy threshold is therefore prone to bias and unstable classification. To address this issue, we design an adaptive energy thresholding strategy that updates the decision boundary online based on observed batch statistics. During each training iteration, the mean and variance of the free energy for known samples are calculated as:

$$\overline{E}={\displaystyle \frac{1}{N}}\sum _{i=1}^N{E}_i,S^2={\displaystyle \frac{1}{N}}\sum _{i=1}^N{{(E}_i-\overline{E})}^2$$

(12)

where $N$ denotes the batch size and $E_i$represents the free energy of the $i$-th sample. Because these batch-level statistics are sensitive to noise and sampling fluctuations, we apply an exponential moving average (EMA) mechanism to stabilize them over time.

Given an update rate $\eta \in \left(0,1\right)$, the global mean and standard deviation at iteration $t$ are updated as:

$${\mu }_t=\left(1-\eta \right){\mu }_{t-1}+\eta \overline{E},{\sigma }_t=\sqrt{\left(1-\eta \right){\sigma }_{t-1}^2+\eta S^2}$$

(13)

This recursive formulation smooths short-term fluctuations while retaining the long-term statistical trend of the overall energy distribution. As a result, it provides a more stable and robust estimate of the global energy landscape, even under dynamic or noisy signal conditions.

Once the global statistics (${\mu }_t,{\sigma }_t$) are obtained, the rejection threshold is determined according to the Gaussian confidence interval principle:

$$\tau ={\mu }_t+\beta {\sigma }_t$$

(14)

where $\beta$ is a hyperparameter controlling the strictness of the boundary. A larger $\beta$ produces a more relaxed threshold, allowing greater tolerance to noise, whereas a smaller $\beta$ enforces stricter rejection, which is advantageous when unknown signals are more likely to appear.

During inference, the deep feature $\mathit{z}$ of each input sample is first extracted, and its free energy $E\left(\mathit{z}\right)$ is computed. Classification is then performed based on the adaptive threshold $\tau$:

$$\widehat{y}=\left\{\begin{array}{c}argmax\left(sim\left(\mathit{z},{\mathit{c}}_k\right)\right),ifE\left(\mathit{z}\right)\le \tau \\ Unknown,ifE\left(\mathit{z}\right)>\tau \end{array}\right.$$

(15)

If the signal originates from a known class, its energy remains below the threshold, and it is assigned to the class with the highest cosine similarity. Conversely, samples far from all class embeddings exhibit higher energy values and are rejected as unknown.

3. Results

3.1. Experimental Setup and Evaluation Protocol

3.1.1. Dataset Description

To evaluate the proposed open-set UAV signal recognition method, we used a self-developed lightweight target intelligent acquisition system to collect image transmission signals from DJI Phantom 4 RTK UAVs. The system covers a frequency range from 1.5 MHz to 8 GHz, has a real-time bandwidth over 30 MHz, and supports 32 simultaneous channels, allowing signals to be captured across multiple frequency bands. Experiments were conducted indoors to ensure signal stability. The receiving antenna was positioned approximately 20 meters from the UAV remote controller. The system’s center frequency was set to 5.8 GHz, and the 32 channels covered from 5.725 GHz to 5.850 GHz. Signals were continuously recorded for 10 seconds at 30 Msps (complex IQ). The data acquisition process is shown in Figure 6.

Ten UAVs of the same model but different individual units were used, each treated as a separate class. All UAVs transmitted signals independently without mutual interference. The UAV image transmission signals use OFDM modulation. To ensure each sample contains enough information and meets model input requirements, we extracted 10 ms segments from the recordings. Each segment was then downsampled to a fixed length of 4096 complex IQ points, followed by direct current removal and normalization. Spectrum and time-domain checks confirmed that signal bands and OFDM frames were intact.

After screening, 5,000 samples were collected (500 per UAV). Although the UAVs share the same protocol and hardware, small manufacturing differences introduce slight variations in RF signals, including oscillator drift, non-linear distortion, and front-end noise. These differences form unique RF fingerprints, which deep neural networks can learn for fine-grained UAV identification. This dataset serves as a benchmark for both closed-set classification and open-set UAV signal recognition.

3.1.2. Experimental Environment and Implementation Details

All experiments were done on a workstation with an Intel Core i9-11950H CPU and an NVIDIA RTX A3000 GPU. The model was implemented using PyTorch 2.2.2 with CUDA 12.1 support. To make the comparison fair, all models were trained and tested on the same dataset partition, using the same random seed. For the known classes, the data were divided into training, validation, and test sets in the ratio of 6:2:2. The detailed hyperparameter settings and implementation configurations are given in Table 1.

Each experiment was repeated several times, and we report the average results to reduce randomness and improve reproducibility.

3.1.3. Evaluation Metrics

To evaluate the model performance for both closed-set classification and open-set recognition, four main metrics were used: Acc, F1-score, AUROC, and OSCR.

Acc: Acc measures the overall proportion of correctly classified known samples and is defined as:

$$Acc={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(16)

where $TP$, $TN$, $FP$, and $FN$ denote true positives, true negatives, false positives, and false negatives, respectively. This metric provides an intuitive assessment of the closed-set classification precision.

F1-score: The F1-score provides a balanced measure between precision and recall, offering a fair assessment of classification performance, especially when the class distribution is uneven. It is computed as:

$$Precision={\displaystyle \frac{TP}{TP+FP}},Recall={\displaystyle \frac{TP}{TP+FN}}$$

(17)

$$F1={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}$$

(18)

AUROC: AUROC quantifies how well the model distinguishes between known and unknown samples. It corresponds to the area under the receiver operating characteristic (ROC) curve, which captures the relationship between the true positive rate (TPR) and the false positive rate (FPR):

$$TPR={\displaystyle \frac{TP}{TP+FN}},FPR={\displaystyle \frac{FP}{FP+TN}}$$

(19)

$$AUROC={\int }_0^{1}TPR\left(FPR\right)d\left(FPR\right)$$

(20)

A higher AUROC value reflects a stronger ability to separate unknown samples from known ones, as well as a more stable decision boundary.

OSCR: OSCR jointly evaluates closed-set classification accuracy and open-set rejection performance, providing a unified view of recognition effectiveness. It is defined as:

$$CCR\left(\theta \right)={\displaystyle \frac{1}{N_c}}\sum _{i=1}^{N_c}1\left({\widehat{y}}_i=y_i,S_i\ge \theta \right)$$

(21)

$$FPR\left(\theta \right)={\displaystyle \frac{1}{N_o}}\sum _{j=1}^{N_o}1\left(S_j\ge \theta \right)$$

(22)

$$OSCR={\int }_0^{1}CCR\left(FPR\right)d\left(FPR\right)$$

(23)

where $N_c$ and $N_o$ denote the number of closed-set and open-set samples, respectively; $S$ represents the model’s confidence score, and $\theta$ is the decision threshold. Higher OSCR values indicate a better overall trade-off between correctly classifying known classes and successfully rejecting unknown ones.

3.2. Closed-Set Recognition Performance

A strong closed-set classification backbone is the prerequisite for reliable open-set recognition. To assess the effectiveness of the proposed feature extractor in terms of ac-curacy, noise robustness, and parameter efficiency, we compared it with several representative IQ-based electromagnetic signal classification models, including ResNet [43], GRU [44], MCLDNN [45], and IQformer [46].

Figure 7 shows the closed-set recognition accuracy of all models across different SNRs. As expected, performance improves as SNR increases. However, the differences between models become pronounced at low SNR (SNR < 0 dB). Under substantial noise interference, both GRU and IQformer show a noticeable drop in accuracy, while ResNet and MCLDNN remain relatively stable but still underperform the proposed model. Across the entire SNR range, our method maintains the highest accuracy, suggesting stronger resilience and better generalization in noisy communication environments.

At an SNR of 0 dB, the detailed results are listed in Table 2.

The proposed model achieves the highest Acc and F1-score among all compared methods, while also maintaining the lowest parameter count (0.062M). This balance indicates that the network can efficiently learn discriminative representations of UAV signals with minimal computational cost—an essential advantage for real-time or embedded anti-UAV systems.

To better understand category-level performance, Figure 8 presents the confusion matrices of all models at 0 dB SNR. The proposed model shows strong diagonal dominance, indicating that most samples are correctly classified. Misclassifications are fewer and less severe compared with the baseline models, consistent with the quantitative results.

To provide a feature-level view, Figure 9 visualizes the learned representations using t-SNE. The proposed model produces compact and well-separated clusters, with samples from the same class grouped tightly and clear margins between classes. In contrast, features from the baseline models show noticeable overlap, indicating weaker inter-class separation. This visualization supports the observed quantitative improvements.

In summary, these results show that the proposed model achieves strong closed-set classification. The learned feature space demonstrates compact intra-class clusters and clear inter-class boundaries, which improves accuracy under noisy conditions and provides a solid basis for the subsequent geometry-energy-based open-set recognition.

3.3. Open-Set Recognition Results

To comprehensively evaluate the effectiveness of the proposed GE-OSR framework in open-set recognition tasks, the performance was compared with several baseline methods, including OpenMax [32], ARPL [37], PROSER [47], CSSR [48], and the state-of-the-art S3R [49]. Experiments were conducted along two dimensions: (1) varying levels of openness, and (2) different SNR conditions.

First, the number of known classes was fixed at four, and the SNR was set to 0 dB to examine how the number of unknown classes affects recognition performance. Table 3 presents the AUROC results for each method under different numbers of unknown classes. As the number of unknown classes increases, AUROC values decrease consistently across all methods, reflecting the growing difficulty of open-set recognition at higher openness levels. Among all methods, GE-OSR achieves the highest AUROC scores and exhibits the smallest decline as openness increases. For instance, with six unknown classes, GE-OSR still reaches an AUROC of 96.39% ± 0.56%, demonstrating strong performance even under challenging conditions. The relatively high average AUROC and small standard deviations indicate that GE-OSR is stable and reliable across different openness settings.

Table 4 shows the OSCR results under the same experimental setup. Similar to AUROC, OSCR decreases gradually as the number of unknown classes grows. However, GE-OSR consistently outperforms other methods in terms of OSCR, maintaining high scores across all openness levels. For example, when six unknown classes are present, GE-OSR achieves an OSCR of 95.39% ± 0.67%, which is higher than all baseline methods.

In the subsequent experiments, the number of known categories was fixed at four and the number of unknown categories at six. Since open-set UAV signal recognition needs to consider both classifying known categories correctly and rejecting unknown ones, OSCR is a more suitable metric as it evaluates both at the same time. Therefore, Figure 10 illustrates the OSCR performance of each method under different SNR conditions, providing a clearer view of how noise affects the open-set classification ability.

As shown in Figure 10, the OSCR performance of all models improves with increasing SNR. At low SNR, the proposed GE-OSR model clearly outperforms the other methods. When the SNR is moderate or high (≥ 0 dB), S3R also performs well, but GE-OSR still achieves higher performance, reaching an OSCR of about 0.96 at high SNR. This means the model has a higher upper limit of performance and is more stable.

To show how different methods separate features, Figure 11 gives t-SNE visualizations for known and unknown classes.

As shown in Figure 11, the features learned by the proposed GE-OSR framework display the most compact intra-class clusters and the clearest inter-class separations. More importantly, unknown samples are well-isolated from known clusters, forming distinct and independent regions in the feature space.

3.4. Impact of Threshold Settings on Open-Set Recognition Performance

Threshold selection is very important in open-set recognition because it affects both the classification accuracy of known categories and the rejection rate of unknown signals. To show how this balance works, Figure 12a presents the free-energy distributions for known and unknown samples. Figure 12b shows how recognition accuracy changes as the rejection threshold varies. These figures help illustrate how the adaptive threshold affects the decision boundary and maintains stable recognition.

As seen in Figure 12a, most of the known samples are located in the low-energy region, while the unknown samples are mainly in the higher-energy region. This shows that there is a clear separation between the known and unknown signals in the energy space. However, some known samples still have relatively high energy. This usually happens when the signal has low SNR, channel distortion, or is close to the class boundary, which makes its features deviate from the class embedding. The dashed line represents the adaptive threshold $\tau ={\mu }_t+\beta {\sigma }_t$, which is approximately in the valley between the two distributions. This threshold can automatically adjust based on the energy distribution, so it does not need to be set manually and still effectively separates known and unknown samples.

Figure 12b shows how the threshold value affects the model’s performance. At the current threshold, the GE-OSR model achieves 92.50% accuracy on known UAV signals and rejects 98.67% of unknown ones. These results show that the proposed method can achieve a good balance between correctly recognizing known classes and rejecting unfamiliar signals. The open-set confusion matrix is shown in Figure 13.

From Figure 13, it can be seen that the proposed model maintains high classification accuracy across all known UAV categories and rejects about 98% of un-known samples. Most of the known classes are clearly separated, but there is still some confusion between UAV2 and the Unknown class. About 28% of UAV2 samples are wrongly rejected as unknown, and about 1% of unknown signals are predicted as UAV2. This kind of mistake may be caused by the high similarity between the feature distributions of UAV2 and some unseen signal types, leading to partial overlap in the embedding space.

Even with this problem, the adaptive energy-thresholding method can still work effectively. It can automatically adjust the decision boundary as the feature distribution changes, ensuring the model’s recognition and rejection abilities remain stable across different noise levels and openness conditions.

3.5. Ablation Studies

To study how each part of the GE-OSR model contributes to the final performance, we did several ablation experiments. Three modules were tested: TFFM, DCEL, and FEAL. These modules were added one by one to observe their individual influence on AUROC and OSCR.

All experiments were conducted under the same conditions and repeated 5 times to improve reliability. The mean and standard deviation of the performance are shown in Table 5. For the version without TFFM, we used a 1 × 1 convolutional layer to maintain the same feature dimension, and all other settings remained unchanged.

From Table 5, we can see that the basic model achieves AUROC of 70.09% and OSCR of 62.35%. After adding TFFM, both indexes increase significantly, by +14.5% in AUROC and +19.7% in OSCR. This shows that time–frequency fusion can extract more useful in-formation and make the model more robust to noise. After adding DCEL, the two indicators increase again (+4.9% AUROC, +6.8% OSCR), indicating that the geometric constraint can bring intra-class features closer and enlarge the inter-class margin. When FEAL is also used, both metrics improve by around 6-7%, indicating that controlling the energy distribution is important for a smoother open-set boundary.

In conclusion, TFFM provides a solid foundation for feature extraction, DCEL helps build better geometric representations across classes, and FEAL clarifies the energy distribution between known and unknown areas. When all three modules are combined, the GE-OSR model demonstrates strong discriminative ability and stable recognition performance across different open-set UAV signal environments.

4. Discussion

The mechanism of GE-OSR has two main parts. The DCEL module improves the compactness of feature distribution by pulling samples of the same class closer to their embeddings, so features from the same UAV become more clustered in the embedding space. At the same time, the FEAL module keeps the energy of known samples within a stable, low range, while unknown samples are pushed to higher-energy areas. When these two parts work together, they can build a clear, stable boundary in the geometry-energy space, which helps the model distinguish between known and unseen UAV signal classes more accurately and robustly.

The multi-scale time-frequency modeling in TFFM and CTBlock helps the network extract features that are both more discriminative and more resistant to noise. As a result, the model can still perform well even under strong noise. At the same time, the model has only about 0.062 million parameters, making it very lightweight and suitable for real-time UAV detection systems, including portable or edge devices often used in smart cities, emergency response, and perimeter surveillance.

However, there are still some limitations. The EMA-based threshold can sometimes be slow to react when channel conditions change quickly. The model may also fail to identify signals that are completely different from all known categories. In addition, the delay still needs to be reduced to improve the system’s performance in real-time tasks. In the future, we plan to improve the threshold method, enhance generalization to unseen signals, and further simplify the model architecture to meet the strict real-time and operational requirements of practical UAV monitoring applications.

Overall, GE-OSR demonstrates high accuracy, clear interpretability, and strong generalization ability. The model maintains very stable performance in open-set recognition, handling low signal-to-noise ratios and a large number of unknown signals with ease, whereas most existing methods often fail under such conditions. These results indicate that the joint constraint of geometry and energy is an effective strategy for managing complex electromagnetic signal environments. The combination of geometry and energy not only supports robust UAV signal recognition but also provides a promising approach for other intelligent perception tasks in open and dynamic environments, such as anti-drone surveillance, autonomous UAV navigation, and urban airspace management.

5. Conclusions

This paper proposes GE-OSR, a UAV signal recognition model that integrates geometric structure and energy control. By combining feature geometry with energy adjustment, GE-OSR achieves more stable and robust decisions, outperforming existing methods by up to 2.5% OSCR under challenging conditions. The model is not only accurate but also lightweight and interpretable, making it suitable for real-time deployment in UAV monitoring and anti-drone systems. The geometry-energy modeling approach introduced here offers a promising direction for future intelligent sensing systems. In particular, its ability to distinguish known and unknown UAV signals in complex electromagnetic environments is directly relevant to low-altitude airspace safety, smart city UAV management, and autonomous drone operations.

6. Patents

This work has resulted in the following patent:

Zhou, F.; Long, Y.; Zhou, H.; Ren, H.; Gao, R.; Li, H.; Yi, Z.; Hu, J.; Wang, W. An Open-Set UAV Signal Recognition Method Based on Class Center Learning. China National Intellectual Property Administration (CNIPA), Application No. CN202510455917.9, Publication No. CN120408403A, Filed on April 11, 2025, Published on August 1, 2025.