Point-HRRP-Net: A Deep Fusion Framework via Bi-Directional Cross-Attention for Robust Radar Object Classification in Remote Sensing

1. Introduction

Radar object classification is critical for remote sensing [1,2]. As a classic data source, one-dimensional High-Resolution Range Profile (HRRP) is characterized by its ease of acquisition, low dimensionality, and efficient processing. [3,4,5].

Early HRRP recognition technologies relied heavily on "hand-crafted features" [6,7]. Researchers attempted to extract feature vectors from HRRP using methods like template matching and statistical models (e.g., Hidden Markov Models) [8,9]. However, restricted by shallow feature extraction and heavy reliance on prior knowledge, these methods often struggle to generalize to unseen orientations. [10,11,12]. The emergence of deep learning has addressed these limitations by enabling automatic feature extraction [13]. Researchers initially applied Convolutional Neural Networks (CNNs) [14] to HRRP, treating them as 1D signals to extract local scattering features. To address the inherent sequential dependencies of radar echoes, scholars subsequently employed Recurrent Neural Networks (RNNs) [15,16] and variants like LSTM [17] and GRU [18]. However, these architectures often fail to capture long-range dependencies [19]. This challenge prompted the adoption of the Transformer [20], which utilizes self-attention to model global context with superior parallel efficiency [21,22]. More recently, the field has witnessed the emergence of advanced architectures: the Conformer [23] integrates CNNs and Transformers to simultaneously capture local and global features, while the Mamba framework [24], based on State Space Models (SSM), offers linear computational complexity. Specifically, 1D-Mamba [25] has shown great potential in efficiently processing long HRRP sequences. However, generating HRRP inevitably collapses 3D structures into 1D signals. This compression causes significant information loss, which severely limits generalization to unseen angles. Existing single-modality approaches fail to fundamentally address the limitation.

To overcome this, scholars have turned to multi-modal fusion to incorporate complementary information. For instance, researchers have combined Synthetic Aperture Radar (SAR) [26,27] with optical images to enable all-weather, high-precision ground target recognition. Others have integrated HRRP with micro-Doppler features to refine the distinction of target motion states [28]. For airborne target identification, studies have explored fusing HRRP with Infrared (IR) images [29,30]. In autonomous driving, fusing millimeter-wave radar with LiDAR is widely adopted [31,32]. However, a fundamental distinction exists: millimeter-wave radar provides 3D spatial data, whereas HRRP consists solely of 1D signatures. Consequently, the fusion of LiDAR and HRRP remains largely unexplored.

Advancements in LiDAR technology [33] have made acquiring 3D LiDAR point clouds from distant targets a reality. Since point clouds preserve intrinsic geometric topology that remains invariant under rigid motion [34], they serve as an ideal candidate for fusion with HRRP. Regarding processing methods, PointNet[35] and PointNet++ [36] pioneered direct learning on unordered points, bypassing inefficient voxelization [37,38,39]. Subsequently, Dynamic Graph CNN (DGCNN) [40] introduced dynamic graph convolution to capture local topology, while Point Transformer [20] and PointMLP [41] introduced self-attention and pure residual designs, respectively. Most recently, the Mamba framework has extended its success to the 3D domain. Specifically, PointMamba [42] adapts the SSM mechanism to point clouds, achieving a superior balance between performance and computational efficiency.

Conversely, HRRP provides scattering information that resolves geometric ambiguities caused by point cloud occlusions. As shown in Figure 1, when the line of sight directly faces the base, the cone, cylinder, and cone-cylinder assembly all appear as identical circles [Figure 1a–c]. This makes them indistinguishable based on shape alone. However, their corresponding HRRP remains distinct due to specific scattering mechanisms like edge diffraction [Figure 1d–f].

Traditional strategies like early and late fusion fail to capture deep feature interactions due to their shallow interaction mechanisms. [43,44]. To address this, the field shifted toward intermediate fusion to enable complex feature interactions [45,46,47]. Specifically, cross-attention mechanisms were introduced for fine-grained feature alignment [48,49]. Concurrently, to alleviate the computational burden of standard attention, efficiency-oriented designs have emerged, including Linear-Attention [50], Efficient-Attention [51], and the recent Mamba architecture [52]. These models utilize linear-complexity mechanisms to accelerate inference. In this work, to establish explicit, point-to-point interactions between HRRP and 3D point clouds for robust recognition, we propose Point-HRRP-Net. This framework leverages a Bi-Directional Cross-Attention (Bi-CA) mechanism to improve classification accuracy and generalization capability.

The main contributions of this paper are summarized as follows:

• HRRP is distinct from millimeter-wave radar. To the best of our knowledge, this is the first framework to fuse HRRP with 3D point clouds for object classification.
• We propose Point-HRRP-Net, a fusion framework that incorporates a Bi-CA mechanism to integrate HRRP and 3D point clouds.
• We have constructed and publicly released a paired point cloud-HRRP dataset through electromagnetic and optical simulation.
• Experimental results in a simulated environment demonstrate that the proposed framework outperforms single-modality baselines. Comprehensive ablation studies validate the design rationale, demonstrating the necessity of both the specific feature extractors and the Bi-CA mechanism for achieving robust performance.

This paper is structured as follows: Section 2 details our proposed method. Section 3 covers dataset creation and experimental results. Section 4 provides a discussion. Finally, Section 5 offers a conclusion.

2. Methods

2.1. Point-HRRP-Net Overview

The overall architecture of our proposed network is illustrated in Figure 2. The data processing pipeline can be conceptually divided into three main stages: (1) dual-branch feature extraction, (2) Bi-CA fusion, and (3) classification.

The process begins by normalizing the HRRP and point cloud inputs. Subsequently, these modalities are processed in parallel by specialized feature extractors. Specifically, the HRRP branch analyzes in both the time and frequency domains using a hybrid CNN-Transformer architecture, while the point cloud is encoded by a DGCNN-based extractor to capture geometric features. The extracted feature sequences are then fused via the Bi-CA mechanism to enable explicit cross-modal interaction. Finally, the fused features are aggregated through pooling and concatenation, and subsequently fed into an MLP for classification.

2.2. HRRP Feature Extractor

As shown in Figure 3, we propose a dual-stream extractor to capture target characteristics from both time and frequency domains. One stream processes the raw HRRP to analyze time-domain scattering distributions, while the other processes the amplitude spectrum (derived via FFT) to encode frequency-domain attributes. The extracted features from both branches are then projected and concatenated into a unified HRRP feature sequence, which serves as the input for the subsequent fusion module.

The time-domain branch processes the raw HRRP directly. Let the input be represented as a vector $\mathbf{x}\in {\mathbb{R}}^L$, where $L=512$ denotes the number of range bins. To capture local scattering patterns, we employ a hierarchical 1D-CNN backbone. This network consists of stacked convolutional blocks, each comprising 1D convolution layers followed by Max-Pooling. This design enables the extraction of features at varying scales, efficiently encoding both detailed peaks and global structural semantics.

The data flow through the time-domain 1D-CNN is detailed in Figure 4. Specifically, the hierarchical architecture comprises three main blocks with increasing channel depths (8, 16, and 32 filters, respectively). The input tensor, with a shape of ($B,1,512$) where B is the batch size, is first processed by two convolutional layers with 8 filters, followed by a Max-Pooling operation that halves the sequence length. This process is repeated with deeper feature maps. The 1D convolution operation at a layer k can be formally expressed as:

$$z_j^{\left(k\right)}=\sigma \left(\sum _{i=1}^{C_{in}}{w}_{ij}^{\left(k\right)}*h_i^{(k-1)}+b_j^{\left(k\right)}\right)$$

(1)

where ${\mathbf{h}}^{(k-1)}$ is the input feature map from the previous layer, ${\mathbf{w}}^{\left(k\right)}$ and ${\mathbf{b}}^{\left(k\right)}$ are the learnable filter weights and biases, * denotes the 1D convolution with padding, and $\sigma$ is the ReLU activation function. The subsequent Max-Pooling operation reduces the dimensionality and provides local translational invariance. After passing through the 1D-CNN backbone, the raw HRRP is transformed into a compact sequence of high-level feature vectors ${\mathbf{Z}}_t\in {\mathbb{R}}^{B\times L^{\prime }\times C_{out}}$, where $L^{\prime }$ is the reduced sequence length and $C_{out}$ is the final channel dimension (32 in this case).

Following local feature extraction by the 1D-CNN, the sequence ${\mathbf{Z}}_t$ is fed into a Transformer Encoder. Unlike 1D-CNNs, which are limited by local receptive fields, the Transformer utilizes multi-head self-attention to capture global dependencies among scattering centers. This mechanism dynamically weights the importance of all elements in the sequence, effectively modeling the target’s global context. The output is an enriched time-domain feature sequence ${\mathbf{H}}_t\in {\mathbb{R}}^{B\times L^{\prime }\times D_t}$, where $D_t=32$.

Concurrently, the second stream processes the frequency-domain information. The amplitude spectrum of the HRRP is first obtained via Fast Fourier Transform (FFT) and taking the absolute value:

$${\mathbf{x}}_f=\left|\mathcal{F}\left(\mathbf{x}\right)\right|$$

(2)

where $\mathcal{F}\left(·\right)$ denotes the FFT operator. This spectrum ${\mathbf{x}}_f$ is then processed by an analogous architecture consisting of a hierarchical 1D-CNN and a Transformer Encoder. For efficiency, we employ a lightweight 1D-CNN architecture ((2, 4), (2, 8), (1, 16)) to extract salient spectral features. This yields a frequency-domain feature sequence ${\mathbf{H}}_f\in {\mathbb{R}}^{B\times L^{\prime \prime }\times D_f}$, where $D_f$ is set to 16.

The final step is Feature Unification. Since the feature sequences ${\mathbf{H}}_t$ and ${\mathbf{H}}_f$ have differing dimensions ($D_t=32$ and $D_f=16$), they are projected into a unified feature space of dimension $D_{seq}=64$ using separate linear layers:

$$\begin{array}{cc}\hfill {\mathbf{H}}_t^{\prime }& ={\mathbf{H}}_t{\mathbf{W}}_t+{\mathbf{b}}_t\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\mathbf{H}}_f^{\prime }& ={\mathbf{H}}_f{\mathbf{W}}_f+{\mathbf{b}}_f\hfill \end{array}$$

(4)

where ${\mathbf{W}}_t,{\mathbf{W}}_f$ are learnable weight matrices and ${\mathbf{b}}_t,{\mathbf{b}}_f$ are learnable bias vectors. This projection ensures dimensional compatibility. Finally, the mapped sequences are concatenated along the length dimension to produce the unified HRRP representation ${\mathbf{H}}_{hrrp}\in {\mathbb{R}}^{B\times (L^{\prime }+L^{\prime \prime })\times D_{seq}}$. This sequence serves as the input for the subsequent fusion module.

2.3. 3D Point Cloud Feature Extractor: DGCNN

To mitigate the aspect sensitivity inherent in HRRP, we utilize point clouds to provide rotation-invariant geometric context. We employ a DGCNN [40] as the feature backbone. Unlike architectures operating on fixed grids, DGCNN dynamically constructs local neighborhood graphs in the feature space, enabling the effective capture of fine-grained topological structures regardless of the target’s pose.

Subsequently, for a given point cloud input with $\mathcal{N}=256$ points, we represent each point ${\mathbf{p}}_i$ by a feature vector (initially its 3D coordinates). At each layer, we construct a local geometric structure by identifying the $k=20$ nearest neighbors (k-NN) for every point. Crucially, this graph is dynamically updated at each network depth, allowing the model to group points based on learned semantic similarities rather than just physical proximity.

The core operation is the EdgeConv block, which computes "edge features" describing the relationship between the central point ${\mathbf{p}}_i$ and its neighbors ${\mathbf{p}}_j$. To capture both local geometry and global position, we formulate the edge feature $e_{ij}$ as:

$${\mathbf{e}}_{ij}=({\mathbf{p}}_j-{\mathbf{p}}_i,{\mathbf{p}}_i)$$

(5)

where $(·,·)$ denotes concatenation. Here, ${\mathbf{p}}_j-{\mathbf{p}}_i$ encodes the local neighborhood structure, while ${\mathbf{p}}_i$ preserves absolute spatial information. These features are processed by a shared-weight Multi-Layer Perceptron (MLP), implemented efficiently as a 1×1 convolution. Finally, a channel-wise symmetric function (Max-Pooling) aggregates information from the local neighborhood, ensuring permutation invariance. This operation is defined as:

$${\mathbf{p}}_i^{\prime }=\underset{j:(i,j)\in \mathcal{E}}{max}\left(h_{\Theta }({\mathbf{p}}_j-{\mathbf{p}}_i,{\mathbf{p}}_i)\right)$$

(6)

where $\mathcal{E}$ represents the set of edges in the dynamically constructed graph, and $h_{\Theta }$ denotes the learnable MLP.

To preserve geometric information across different abstraction levels, we aggregate the outputs from all stacked EdgeConv blocks. Specifically, the intermediate feature maps are concatenated along the channel dimension. This skip-connection design effectively integrates fine-grained local details from shallow layers with global semantic contexts from deep layers.

Following feature aggregation, a shared MLP (implemented as a 1D convolution) projects the combined features into a high-dimensional embedding space. The final output of the point cloud branch is a feature sequence ${\mathbf{H}}_{pc}\in {\mathbb{R}}^{B\times \mathcal{N}\times D_{pc}}$, where $\mathcal{N}$ is the number of points and $D_{pc}=1024$. This sequence provides the rotation-invariant geometric representation required for the subsequent Bi-CA fusion module.

2.4. Bi-CA Fusion Module

To effectively fuse the heterogeneous HRRP and point cloud features, we introduce a Bi-CA mechanism. This approach allows each modality to dynamically query the other, thereby selectively integrating complementary information.

Prior to fusion, we must map the input features into a shared latent space. Let the output sequence from the point cloud extractor be denoted as ${\mathbf{H}}_{pc}\in {\mathbb{R}}^{B\times \mathcal{N}\times D_{pc}}$ (where $D_{pc}=1024$), and the unified sequence from the HRRP extractor as ${\mathbf{H}}_{hrrp}\in {\mathbb{R}}^{B\times (L^{\prime }+L^{\prime \prime })\times D_{seq}}$ (where $D_{seq}=64$). To align these features within a shared semantic space, we project both modalities to a common dimension $D_{fusion}=64$ using separate learnable linear layers.

Formally, this projection is defined as:

$$\begin{array}{cc}\hfill {\mathbf{H}}_{pc}^{\prime }& ={\mathbf{H}}_{pc}{\mathbf{W}}_{pc}+{\mathbf{b}}_{pc}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {\mathbf{H}}_{hrrp}^{\prime }& ={\mathbf{H}}_{hrrp}{\mathbf{W}}_{hrrp}+{\mathbf{b}}_{hrrp}\hfill \end{array}$$

(8)

where ${\mathbf{W}}_{pc}\in {\mathbb{R}}^{D_{pc}\times D_{fusion}}$ and ${\mathbf{W}}_{hrrp}\in {\mathbb{R}}^{D_{seq}\times D_{fusion}}$ are learnable weight matrices, and ${\mathbf{b}}_{pc}$ and ${\mathbf{b}}_{hrrp}$ are the corresponding bias vectors, where ${\mathbf{H}}_{pc}^{\prime }$ and ${\mathbf{H}}_{hrrp}^{\prime }$ are the aligned feature sequences serving as inputs for the fusion module. As illustrated in Figure 5, the first stream utilizes the aligned point cloud features ${\mathbf{H}}_{pc}^{\prime }$ to generate the Query vectors ${\mathbf{Q}}_{pc}$, while the HRRP features ${\mathbf{H}}_{hrrp}^{\prime }$ are projected to produce the Key ${\mathbf{K}}_{hrrp}$ and Value ${\mathbf{V}}_{hrrp}$ vectors. The attention mechanism dynamically aggregates electromagnetic information for each geometric point:

$$\begin{array}{c}\hfill \mathrm{Attention}(Q_{pc},K_{hrrp},V_{hrrp})=\mathrm{Softmax}\left({\displaystyle \frac{Q_{pc}{K}_{hrrp}^T}{\sqrt{d_k}}}\right)V_{hrrp}\end{array}$$

(9)

The geometric representation is then updated via a residual connection and Layer Normalization:

$$\begin{array}{c}\hfill H_{pc}^{\prime \prime }=\mathrm{LayerNorm}\left(H_{pc}^{\prime }+\mathrm{Attention}(Q_{pc},K_{hrrp},V_{hrrp})\right)\end{array}$$

(10)

This process effectively enriches the point cloud representation by selectively integrating complementary HRRP cues, yielding more discriminative geometric features. Symmetrically, the HRRP-enrichment stream employs ${\mathbf{H}}_{hrrp}^{\prime }$ as the Query and ${\mathbf{H}}_{pc}^{\prime }$ as the Keys and Values to ground abstract radar signals into the 3D geometric context, following the same formulation as Equation (9) and (10).

To facilitate deep feature interaction, we stack $L=3$ layers with 8 attention heads, a choice validated by the sensitivity analysis in Appendix Figure A1. The refined feature sequences are aggregated via Global Average Pooling (GAP) and concatenated to form a unified vector ${\mathbf{v}}_{final}\in {\mathbb{R}}^{B\times 2D_{fusion}}$, which is fed into an MLP classifier. Regarding the attention configuration, we adopt a Post-Norm design (normalization after residual connection) to ensure training stability and forgo causal masking to maintain a global receptive field for effective bidirectional modeling.

2.5. Experimental Setup and Implementation Details

All model training and accuracy evaluations were conducted on a Windows platform equipped with an NVIDIA RTX 5070 GPU (Blackwell Architecture).

To ensure fair efficiency comparisons, we evaluated inference latency, FLOPs, and parameter counts on a Linux workstation powered by an NVIDIA RTX 4090. This hardware transition was necessitated by the Mamba-based baselines, which rely on the mamba-ssm library’s optimized CUDA kernels. These kernels require specific environment configurations (Linux OS and mature CUDA versions) that currently face compatibility constraints on the RTX 50 series architecture.

For model optimization, we employed the Adam optimizer with a weight decay of 1e-5. A differential learning rate strategy was adopted to accommodate the distinct characteristics of different network components. Specifically, the learning rates for the HRRP feature extractor and the point cloud feature extractor were set to 3e-4 and 5e-4, respectively. The remaining modules, including the feature mapping layers, the cross-attention layers, and the final classifier, were assigned a learning rate of 1e-4. All models were trained with a batch size of 32. For a fair comparison, we employed an early stopping protocol to ensure convergence. The training process was terminated if the validation loss did not decrease for 20 consecutive epochs. The model checkpoint corresponding to the lowest validation loss was then selected for the final evaluation.

3. Results

3.1. Dataset Setup

3.1.1. Target Geometry and Parameters

Direct acquisition of measured electromagnetic data is difficult [53,54]. Our research requires a dataset containing paired point cloud and HRRP data. However, existing public datasets are often mismatched in terms of objects, environments, or viewpoints [55,56,57]. It is practically infeasible to generate simulated data for another modality that pairs with existing real-world samples. To address this issue, we constructed a dataset specifically for this task through high-fidelity electromagnetic and optical simulations.

We selected three representative classes of Perfect Electrical Conductor (PEC) targets. Following precedents in related literature on object classification [58], our selection comprises cones, cylinders, and composite objects formed by an upper cone and a lower cylinder. These targets serve as fundamental geometric primitives of more complex objects, providing a foundational and challenging test case for recognition algorithms. Furthermore, their scattering characteristics cover a wide range of typical electromagnetic phenomena—including scattering from tips, edges, curved surfaces, and geometric discontinuities—thus making them broadly representative for robustness evaluation.

The geometric configuration and dimensional parameters of the targets are illustrated in Figure 6 and Figure 7. Figure 6 defines the observation geometry, illustrating the coordinate system and the incident angle, $\theta$, which represents the line-of-sight direction relative to the target’s primary axis. The detailed longitudinal cross-sections and specific dimensions of these targets are provided in Figure 7. To ensure numerical stability and avoid meshing singularities during the electromagnetic simulation, the mathematically sharp tips of the cone and the cone-cylinder composite were regularized by replacing them with small paraboloids. The paraboloid for the single cone target was defined with a focal length of 0.0025 m, while that of the composite object’s tip utilized a focal length of 0.004 m.

3.1.2. Multimodal Data Simulation

1D HRRP Data: We employed the electromagnetic simulation software FEKO to generate the HRRP data. In the simulation, the target surfaces were set as PEC, and a wideband signal with a center frequency of 6 GHz and a bandwidth of 4 GHz (ranging from 4 to 8 GHz) was used for excitation. The signal was a stepped-frequency waveform with a frequency step of 50 MHz.

The theoretical range resolution, $\Delta R$, is determined by the signal bandwidth, B, according to

$$\Delta R={\displaystyle \frac{c}{2B}},$$

(11)

where c is the speed of light. The maximum unambiguous range, $R_{\mathrm{un}}$, is determined by the frequency step, $\Delta f$, as

$$R_{\mathrm{un}}={\displaystyle \frac{c}{2\Delta f}}.$$

(12)

According to Equations (11) and (12), the calculated range resolution is 3.75 cm, which is sufficient to capture the fine structural information of the targets, and the unambiguous range is 3 m, which completely encompasses the targets with an adequate margin. Considering that all three target types are bodies of revolution, we sampled them uniformly along the elevation angle ($\theta$) from 0° to 180° at 1° intervals, generating a total of 543 raw HRRP samples (181 angles for each of the three target types).

We designed a Python algorithm to generate the 3D point clouds of the targets by simulating LiDAR illumination on their 3D surfaces and capturing the reflected sparse point clouds. The viewpoints for point cloud generation were strictly aligned with the observation angles of the HRRP simulations. The raw point clouds were then uniformly down-sampled to a set of 256 points, resulting in 543 raw point cloud samples that are strictly paired with the HRRP.

3.1.3. Data Augmentation and Dataset Splitting

Data Augmentation and Scaling: To enhance model robustness and expand the dataset, we designed a joint data augmentation scheme comprising 15 distinct strategies, such as Gaussian noise injection, coordinate jittering, and global scaling. Detailed configurations for each strategy are provided in Appendix A (Table A1 ). Through these operations, the original 543 data pairs were expanded to a total of 8,688 samples (543 × 16). Prior to training, all HRRP data underwent Min-Max normalization, while point clouds were centered and normalized to fit within a unit cube.

Dataset Splitting: To rigorously evaluate generalization capability to unseen viewpoints, we adopted a structured, angle-based splitting strategy. The observation range from 1° to 180° was partitioned into contiguous, non-overlapping angular blocks. Within each block, data were divided into training, validation, and test sets according to a 5:2:2 ratio. To systematically assess robustness against varying degrees of angular separation, we established six experimental configurations with block sizes ranging from 9°to 180°.

For instance, in the 90° split configuration, the 180° observation range is divided into two blocks (i.e., 1°–90° and 91°–180°). For the first block, the 5:2:2 ratio allocates angles 1°–50° to training, 51°–70° to validation, and 71°–90° to testing. A similar division is applied to the second block. A larger block size imposes greater angular separation, presenting a more challenging generalization task. The sample corresponding to the 0° observation angle was consistently included in the training set across all configurations. Supplementary analysis (Appendix D, Figure A2) indicates that excluding $0^{\circ }$ samples causes a negligible accuracy drop (< 0.8%), ruling out potential data leakage from back-scattering symmetry.

3.1.4. Evaluation Metrics

To comprehensively evaluate the classification performance, we employ two standard metrics: Overall Accuracy (OA) and F1-Score. Accuracy serves as the primary metric for the general performance comparisons presented in Table 1. Additionally, given the potential for geometric confusion between target classes at specific viewpoints, we utilize the F1-Score (the harmonic mean of Precision and Recall) in our ablation studies (Table 2 and Table 3). This metric provides a more robust assessment of the model’s ability to balance precision and recall, ensuring that the performance improvements are not biased towards specific classes.

We also evaluate its efficiency from three perspectives:

Parameters (Params): We calculate the number of trainable parameters of the entire model, measured in millions (M), to quantify the model’s size and memory footprint.

FLOPs (G): This metric quantifies the number of operations required for a single forward pass, measured in GFLOPs.

Latency (ms): Latency metrics in this paper represent the inference time of a single sample (batch size = 1). Unless otherwise specified, results are reported on an NVIDIA GeForce RTX 4090 GPU.

We conducted comprehensive inference latency benchmarks across a wide range of hardware to assess deployment feasibility. As detailed in Table A2, our tests spanned from data center accelerators to consumer-grade GPUs.

For edge deployment, we specifically utilized the NVIDIA Jetson Orin Nano (8GB). The experimental environment on this embedded platform was configured with Ubuntu 22.04 LTS and CUDA 12.6.68. Operating under the 15 W power mode, the embedded platform achieved an average inference latency of 33.29 ms. This result indicates that the proposed model possesses favorable deployment capabilities, with acceptable inference speeds on embedded platforms.

3.2. Experimental Results

In this section, we evaluate Point-HRRP-Net from three perspectives: (1) comparison with single-modality baselines to assess generalization; (2) ablation studies on fusion strategies to validate the cross-attention mechanism; and (3) analysis of the feature extractor to justify our architectural choices.

3.2.1. Performance Comparison against Single-Modality Methods

Table 1 presents the generalization performance comparison of Point-HRRP-Net against eight representative single-modality methods under six angle-based split configurations. These include our baseline HRRP network (HRRP-only), the MSDP-Net, Point-Transformer, and DGCNN, as well as the recent advanced Transformer-style network (Conformer) and Mamba-style networks (1D-Mamba and PointMamba). A vertical comparison of accuracy reveals that the multi-modal framework consistently outperformed single-modality methods across both small and large angle dataset splits. Specifically, under the $9^{\circ }$ split configuration, our multi-modal model achieved a peak accuracy of 97.51%.

As the angular separation of the dataset splits increased from $9^{\circ }$ to ${180}^{\circ }$, the overall recognition performance of all models exhibited an expected downward trend. A horizontal comparison indicates that Point-HRRP-Net demonstrated excellent robustness under large viewpoint changes. In the ${180}^{\circ }$ split, Point-HRRP-Net maintained an accuracy of 57.67%, surpassing all baseline models. It outperformed the baseline HRRP-only by 12.05% and the best-performing point cloud baseline, PointMamba, by 3.87%.

Regarding the single-modality HRRP baselines, performance variations among different architectures were pronounced. Leveraging its powerful sequence modeling capabilities, Conformer achieved the best results among HRRP-based methods on our dataset. For instance, it attained 52.45% accuracy under the ${180}^{\circ }$ split, outperforming both MSDP-Net and 1D-Mamba. Notably, 1D-Mamba showed a distinct advantage on the 90° split, achieving the highest accuracy among HRRP-based methods in this configuration.

Regarding single-modality point cloud methods, PointMamba significantly outperformed the traditional Point-Transformer and DGCNN across all angle splits. This success stemmed from the Mamba architecture’s advantages in long-sequence and global feature extraction. It achieved an accuracy of 53.80% under the ${180}^{\circ }$ split. This result demonstrated PointMamba’s advanced performance and potential.

3.2.2. Ablation Study on Fusion Strategies

To verify the effectiveness of the proposed Bi-CA mechanism, we conducted comparative experiments by replacing this module with eight fusion strategies: Addition, Product, Gating, Self-Attention, Linear Attention, Efficient Attention, and Bi-Mamba. Detailed ablation study results are listed in Table 2.

For the simple fusion strategies (Concatenation, Addition, and Product), experiments showed they maintained a low inference latency between 5.34 ms and 6.00 ms. However, they performed poorly in large-angle scenarios. These methods achieved respectable F1 scores of approximately 93-96% on the 9° split. While this suggests that basic aggregation is viable when viewpoint discrepancies are minimal, their performance declined significantly as the disparity expanded to 45° and 90°. On the 90° split, Concatenation, Product, and Addition achieved F1 scores of 51.77%, 54.69%, and 60.26%, respectively. Even the best among them lagged behind our proposed method by over 5%. This performance indicates that although simple strategies are fast and parameter-efficient, they lack sufficient capacity to fit complex features. Consequently, they suffer from severe robustness deficiencies under extreme conditions.

Regarding the classic Gating and standard Self-Attention mechanisms, Gating introduced weight modulation but only achieved an F1 score of 58.44% on the 90° test set, failing to surpass the 60% threshold. Self-Attention demonstrated high accuracy under the simple 9° angle but suffered a marked decline on the 90° split, with the F1 score dropping to 47.81%. This represents a gap of 17.73% compared to our method (65.54%). Although Self-Attention theoretically possesses strong fitting capabilities due to its large parameter count, this sharp decline suggests that overfitting occurred in this task. Furthermore, the model complexity increased the latency to 6.99 ms, yet failed to provide the expected accuracy gains.

Subsequently, we evaluated fast attention mechanisms: Linear-Attention and Efficient-Attention. Standard Attention scales quadratically ($\mathcal{O}\left(N^2\right)$) with sequence length. In contrast, these fast variants reduce the cost to complexity $\mathcal{O}\left(N\right)$ through approximation techniques. Experiments showed that while these methods maintained F1 scores of 91-94% on the 9° split, their performance degraded significantly on the 90° split, dropping to 53.70% and 48.54%, respectively. This indicates that while approximation reduced complexity, it also failed to preserve detailed feature information. Moreover, with latencies ranging from 7.6 ms to 8.1 ms, they offered only a marginal speed advantage over our method (8.76 ms).

Finally, we compared the Bi-Mamba fusion strategy. As a representative SSM, the Mamba architecture typically demonstrates superior parameter efficiency and inference speed in long sequence modeling. However, results indicated that Bi-Mamba performed suboptimally in this task. While achieving an F1 score of 95.68% on the 9° split, its score plummeted to 46.09% on the 90° split. This underperformance is likely because Mamba’s linear complexity advantage is maximized in ultra-long sequences, whereas our HRRP (length 512) and point cloud (256 points) data are relatively short sequences. Consequently, Mamba failed to leverage its long-range dependency capabilities. Furthermore, Mamba lacks the explicit token-to-token interaction inherent in attention mechanisms, which resulted in ineffective feature fusion. Bi-Mamba’s inference latency was 8.53 ms, only a negligible advantage over our method (8.76 ms).

Through bi-directional interaction modeling, our method achieved an F1 score of 65.54% on the 90° split, outperforming the second-best method (Addition) by 5.28% and the Self-Attention mechanism by 17.73%. Although this mechanism increased inference latency to 8.76 ms (an increase of approximately 3.4 ms compared to the fastest simple fusion), we consider this latency cost acceptable for radar engineering applications facing complex and dynamic environments.

3.2.3. Ablation Study on Feature Extractors

We conducted a comprehensive ablation study on feature extractors to validate the rationale behind our final architecture. The experimental results are presented in Table 3. By substituting different components, we evaluated the contribution of each part to the overall performance and efficiency.

Regarding HRRP feature extraction, our proposed dual-domain CNN-Transformer demonstrated the best overall performance, achieving an F1-score at 97.47% on the $9^{\circ }$ split. Experimental data showed that 1D-Mamba minimized system latency to 6.52 ms, leveraging its unique selective scan mechanism. However, this speed advantage showed limitations when dealing with large viewing angles. Its F1-score on the ${90}^{\circ }$ split was only 50.10%, lower than our model’s 65.54%. Similarly, the Transformer-based Conformer performed well in sequence modeling (58.02% F1-score on ${90}^{\circ }$). Yet, it failed to outperform our architecture. Traditional recurrent neural networks (RNN, LSTM, etc.) performed the worst as they struggled to capture complex spatial scattering features in HRRP. Therefore, the CNN-Transformer was the robust choice for balancing high F1 scores and real-time performance in this study.

For point cloud feature extraction, we compared DGCNN against classic methods and emerging lightweight architectures. Experimental results revealed a notable discrepancy between theoretical and actual efficiency. Although PointMamba possessed extremely low theoretical FLOPs (only 0.0871 G), its actual inference latency in the multi-modal framework (13.81 ms) was higher than our method (8.76 ms). This indicates that the theoretical efficiency of the Mamba architecture failed to materialize as actual inference acceleration within the current framework. We attribute this discrepancy to three primary factors: First, point clouds are unordered sets. To utilize the SSM, PointMamba requires point ordering. This index reordering of unstructured data on the GPU can disrupt memory coalescence, leading to latency overhead. Second, the Mamba kernel is still under active development, and its underlying CUDA kernel optimization may not be as mature as DGCNN. Third, Mamba’s linear complexity advantage only becomes significant with very long sequences. In this experiment, the point cloud contained only 256 points, rendering the latency optimization insignificant. Regarding generalization performance, DGCNN maintained a leading position. Its F1-score on the ${90}^{\circ }$ split (65.54%) was significantly better than PointMLP (54.90%) and PointMamba (57.19%). This result demonstrated the advantage of DGCNN’s EdgeConv operation in capturing the local geometric structure of 3D targets.

Finally, it is worth noting that while PointMamba showed strong potential in single-modality tasks (as shown in Table 1), its accuracy dropped within our multi-modal fusion framework. This suggests a challenge in heterogeneous feature alignment when fusing PointMamba-generated features with HRRP features. We attribute this decline not to a lack of information, but to the structural incompatibility introduced by serialization. PointMamba flattens 3D structures into 1D sequences, creating a structural mismatch that complicates alignment with HRRP representations. Given the comprehensive advantages of DGCNN in feature robustness, fusion gain, and inference speed, we selected DGCNN as the point cloud feature extractor for this study.

4. Discussion

4.1. Visual Analysis of Cross-Modal Interactions

We visualized the attention weights in Figure 8 to examine how the two modalities interact. In the heatmaps, yellow regions denote higher attention weights, whereas blue regions indicate lower weights.

When point cloud features serve as the query (Figure 8a,b), the model assigns higher weights to high amplitudes in the HRRP time domain and low-to-mid frequencies in the frequency domain. These high-amplitude peaks correspond to the dominant scattering centers of the object. Notably, extremely low frequencies receive almost no weight. This suggests that even if these components possess high spectral amplitudes, the model does not consider them useful for target classification. Physically, extremely low frequencies correspond to the slowest variations in the range profile structure, representing merely coarse outlines. This indicates that the network can effectively identify and utilize dominant scattering features while suppressing clutter from non-informative low-frequency components.

Conversely, when HRRP features are used as the query (Figure 8c), attention is not uniformly distributed. Instead, it focuses on geometric edges and discontinuities. Compared to smooth surfaces, edges typically provide more discriminative information for classification. It appears that the model has learned to focus on these geometrically significant regions. By weighting these features more heavily, the model achieves more stable classification performance. This explains why the cross-attention mechanism is more robust than simple feature concatenation.

4.2. Sim-to-Real Analysis

4.2.1. Analysis of Rotational Offset Scenarios

In real-world applications, multi-modal inputs are rarely perfectly aligned. To quantify the impact of such imperfections, we conducted a misalignment test using the pre-trained model on the full test set. Specifically, we introduced a rotational offset $\Delta \theta$ ranging from $0^{\circ }$ to ${90}^{\circ }$ to one modality during the testing phase to simulate varying degrees of rotational offset.

Figure 9 illustrates the results under the ${18}^{\circ }$ split configuration. We observe that when the rotational offset is within the range of $0^{\circ }$ to ${10}^{\circ }$, the accuracy decline is negligible. Subsequently, the accuracy exhibits slight fluctuations but remains consistently above $98\%$. This suggests that since both modalities describe the same target, the model retains sufficient discriminatory information even with minor misalignments. When the error exceeds ${50}^{\circ }$, the fluctuations increase and a noticeable decline occurs, which can be attributed to feature conflicts arising from significant discrepancies between the modalities. Nevertheless, the accuracy remains above $96.2\%$.

Figure 10 compares the performance across different dataset split configurations. The results for the $9^{\circ }$, ${18}^{\circ }$, ${36}^{\circ }$, and ${45}^{\circ }$ splits are similar, showing a generally flat trend with minimal degradation. However, for the ${90}^{\circ }$ split, the decline in accuracy becomes pronounced as the rotational offset increases. In this configuration, the angular separation between the training and test sets is maximal, placing the highest demand on the model’s generalization capability. We infer that the model likely relies on consistent geometric-physical correspondences for inference in these unseen views. When this correspondence is disrupted by misalignment, the performance drop is consequently more significant compared to less challenging configurations.

4.2.2. Model Stress Test Analysis

To quantify the gap between simulation and real-world data to a certain extent, we conducted progressive stress tests on the model trained with the $9^{\circ }$ split. Specifically, we divided the stress into four scenarios: (1) HRRP SNR deterioration; (2) Random Point Cloud jitter; (3) Random Point Cloud point dropping; (4) The summation of the previous three scenarios. Therefore, we tested these four conditions and observed the accuracy decline curves.

The test results are shown in Figure 11. The first row on the x-axis represents the Signal-to-Noise Ratio (SNR) of the HRRP, decreasing gradually from 30dB. When the HRRP SNR is -15dB, we consider the signal to be almost obscured by noise. The second row represents the magnitude of random point cloud jitter, whose numerical value directly applies to the normalized point cloud coordinates. The third row represents the degree of random point cloud loss. When the point cloud loss reaches 70%, we can consider the point cloud to be almost unrecognizable. Curves of different colors represent the addition of different noise conditions, and the maroon curve represents the superposition of all three conditions.

The y-axis represents the average accuracy. If noise is added only to the HRRP, even if the noise almost overwhelms the signal, the accuracy can still reach 95%. This suggests that the model may rely more on the geometric branch of the point cloud. However, a notable phenomenon is that even if the HRRP is already buried in noise, the multi-modal accuracy is still higher than the single-modality accuracy in Table 1. We speculate that this is because, during the training process, the point cloud has already learned more robust features through the HRRP. The accuracy decline caused by point cloud jitter and point cloud loss is more severe. We speculate that this is because DGCNN is adopted as the point cloud extraction branch, which is relatively sensitive to the geometric relationship between points and their neighbors.

When the three degradation conditions are superimposed (solid maroon line), the model’s accuracy changes very little within the range from the initial state (SNR=30dB, $\sigma =0.01$, Drop=5%) down to the intermediate state (SNR=15dB, $\sigma =0.05$, Drop=20%). This indicates that the system possesses a certain degree of noise resistance. Subsequently, as the severity increases, the performance gradually declines. However, even under the most extreme condition (SNR=-15dB, $\sigma =0.15$, Drop=70%), the accuracy remains at 66.2%. This indicates that the model did not collapse. We believe that although the effective information available is limited, the model remains capable of making valid classification predictions.

Through this experiment, we can quantify the impact of different levels of noise on model performance. It helps us predict the model’s inference ability on real-world data.

4.3. Limitations and Future Directions

Although the proposed method demonstrates favorable performance in experiments, we acknowledge several primary limitations in the current work.

First is the "Sim-to-Real" gap. Current model validation relies on high-fidelity electromagnetic simulation data. This simulation environment is idealized with a clutter-free background and target materials are simplified as Perfect Electrical Conductors (PEC); and the targets consist only of basic geometric shapes (e.g., cones and cylinders). Furthermore, real-world objects may exhibit non-rigid deformations, and the current rigid-body assumption may limit the model’s effectiveness in dynamic scenarios.

Second, regarding robustness, stress test results indicate that the model tends to rely on the geometric features provided by the point cloud for decision-making. When point cloud quality degrades severely (e.g., due to heavy rain or dense fog causing LiDAR failure), the model’s recognition performance declines significantly.

Third, the system requires strict synchronization. Point-HRRP-Net is a strict end-to-end model, requiring input data to be fixed in dimension and strictly paired. However, in practical sensor systems, radar and LiDAR sampling rates are asynchronous. The current architecture does not yet support asynchronous inputs, nor does it possess inference capabilities when a single modality is missing.

Fourth, a scalability bottleneck exists within the cross-attention mechanism. While the model meets real-time requirements at the current data resolution, we must acknowledge that the computational complexity of cross-attention scales quadratically with sequence length. In more complex real-world scenarios, such as those requiring the processing of high-density point clouds or ultra-high-resolution HRRP, the model’s memory consumption and computational load will increase exponentially.

Finally, we identified a compatibility issue with SSM. While PointMamba demonstrated superior performance in single-modality tasks, its accuracy degraded within our framework. The structural incompatibility between serialized 1D features and HRRP representations hinders the effective fusion of these two modalities.

Addressing these limitations, our future research plan focuses on the following directions: (1) Future work will attempt to use real-world measured data for training and testing; (2) We will investigate new training strategies or loss functions to reduce the over-reliance on a single modality; (3) We will improve the model framework to adapt to the asynchronous sampling rates of radar and LiDAR, and explore single-modality inference mechanisms; (4) We will attempt to introduce lightweight techniques, such as efficient attention variants or model pruning, to balance efficiency and accuracy for resource-constrained scenarios. (5) Given the potential demonstrated by PointMamba in single-modality comparisons, we plan to fuse this information for radar object classification in future work.

5. Conclusions

Radar object classification tasks often suffer from the inherent aspect sensitivity of HRRP. To address this, we propose Point-HRRP-Net, a multi-modal fusion framework. Our method leverages the rotational invariance of 3D point clouds to provide stable geometric information. Through a Bi-CA mechanism, we achieve deep feature interaction, effectively alleviating the limitations of aspect sensitivity. Under rigorous angle-based split testing, the model demonstrates robustness and generalization capabilities that surpass single-modality approaches. However, the gap between our simulation and real-world environments must be acknowledged. Our tests in Section 4.2.1 and Section 4.2.2 demonstrate that model accuracy exhibits a decline under simulated rotational offsets, noise, and instability. Moreover, unforeseen factors in physical environments may introduce additional uncertainties. Consequently, we explicitly caution that the present results are confined to simulation, and a performance degradation of ≥ 10% on real data is conceivable. Future work will aim to bridge the "sim-to-real" gap using measured data. Additionally, we plan to explore optimization schemes for asynchronous inputs and lightweight deployment.