Attention-Based Enhancement of Airborne LiDAR across Vegetated Landscapes using SAR and Optical Imagery Fusion

1. Introduction

Accurate three-dimensional (3D) vegetation structure information at submeter spatial scales now plays a key role in applications ranging from wildfire risk modeling [1] to biodiversity and habitat assessment [2,3]. The way vegetation is arranged—its height, density, and continuity—directly influences both fire hazard and fire behavior, and impacts how species use the landscape. Fuels, as opposed to topography and weather, are the only element of the fire behavior triangle that land managers can directly manipulate [4], making structural data vital for strategic interventions. Simultaneously, vegetation structure governs microclimate, resource availability, and landscape connectivity, making it a cornerstone of ecological monitoring and conservation planning [5,6,7]. Airborne light detection and ranging (LiDAR) has emerged as a premier tool for capturing this structural complexity, enabling detailed, landscape-scale mapping of vegetation structure that was previously unattainable with passive optical imagery [2,8,9].

National mapping programs such as the U.S. Geological Survey’s 3D Elevation Program (3DEP) now collect LiDAR data over large areas, but these surveys have important limitations. Typical 3DEP acquisitions are performed at modest point densities (on the order of 0.520pts/$\mathrm{m}$) [10] and with much of the national LiDAR baseline acquired over an extended period (e.g., roughly 2015-2023), a significant portion of this data is now several years old, a situation exacerbated by the lack of a guaranteed or universal update timeline [11]. Consequently, the available point clouds often reflect conditions from several years prior and are relatively sparse compared to those obtained from other platforms (e.g., uncrewed aerial vehicles, or UAVs, see Figure 1).

Critical changes in vegetation structure, such as disturbance-driven loss or ongoing vegetation growth, may go undetected between LiDAR acquisition cycles. This sparsity and temporal gap limit the utility of national LiDAR datasets for applications that require up-to-date, high-resolution 3D information.

Given the limitations of national LiDAR in spatial and temporal resolution, one promising avenue is to enrich these sparse datasets using co-registered imagery from other remote sensing platforms that offer more frequent updates. Sub-meter resolution aerial imagery—such as orthophotos from the National Agriculture Imagery Program (NAIP) [12]—provides fine detail on canopy textures, gaps, and vegetation color, typically acquired at 2 year intervals. Complementing this, L-band synthetic aperture radar known for its sensitivity to vegetation structure [13], can provide valuable multi-temporal data through repeat acquisitions. NASA’s UAVSAR [14], for example, conducts roughly bi-annual L-band campaigns to monitor movement along the San Andreas fault in California, and the upcoming NISAR satellite mission will offer global L-band SAR at a 12-day revisit rate [15]. Fusing such temporally rich optical and radar imagery with existing LiDAR has the potential to produce a denser 3D point cloud reflecting more current vegetation conditions—a challenge well-suited to data-driven approaches such as deep learning. In particular, attention-based models offer a powerful way to integrate these diverse inputs by modeling their spatial and semantic relationships.

Attention mechanisms, first introduced for language translation by [16], enable a model to dynamically determine which parts of the input data are most relevant to each other, a capability crucial for understanding complex scenes. For example, in point clouds of vegetated landscapes, a point on a tree’s leaf can learn, through self-attention, to connect more strongly with its own trunk or branches than with foliage from an adjacent, albeit closer, tree. This ability to discern intrinsic structural relationships could be particularly effective in natural vegetation, as its fractal and self-similar nature provides consistent patterns for self-attention to model across different scales [17,18]. When fusing data, cross-attention extends this by allowing features from one modality, such as a LiDAR point, to selectively query information from another modality, like relevant shadow patterns or canopy gaps identified in NAIP imagery or radar data. These powerful attention operations are the fundamental building block of the influential Transformer architecture [19], which serves as the foundation for nearly all large language model architectures in use today. Building on that success, Transformers were adapted for vision tasks [20] and are now increasingly used across many remote sensing tasks [21]. While these advancements showcase their broad utility, their specific application and optimal adaptation for enhancing sparse airborne LiDAR in complex vegetated landscapes present unique challenges and open questions.

Consequently, key knowledge gaps remain. First, most prior work on data-driven LiDAR enhancement has focused on enhancing point-cloud-derived metrics in raster form (e.g. canopy height [22,23], above-ground biomass [24], elevation [25], and other fuel/vegetation metrics [26,27]) rather than directly enhancing the point cloud itself. Although one recent study successfully upsampled mobile laser scanner point clouds in a forested environment using terrestrial LiDAR as the reference dataset [28], both sensors differ substantially from airborne systems in scale and occlusion behavior. Second, existing deep learning frameworks for point cloud upsampling have primarily been developed and tested on synthetic shapes or man-made objects, and their efficacy on the complex, irregular structures of natural vegetation is not well understood. Third, we found no studies that have attempted to leverage optical or radar imagery for enhancing point clouds in vegetated landscapes. Fourth, we found no studies that have analyzed model performance when the LiDAR input is temporally misaligned with the reference dataset, confounding performance metrics with real-world landscape changes. Deep models are typically trained and evaluated on static scenes, often using an artificially down-sampled point cloud as the input. Thus, it remains unknown how upsampling errors behave in areas where substantial canopy growth or loss has occurred since the original LiDAR survey, or whether multi-modal inputs can mitigate errors stemming from such changes.

1.1. Background and Related Work

1.1.1. Point Cloud Upsampling with Deep Learning

In computer vision and graphics, a range of neural frameworks have been proposed to densify sparse point clouds. PU-Net [29] pioneered the task with multi-layer perceptron (MLP) feature extraction and a point-set expansion module, achieving good fidelity on synthetic computer-designed (CAD) objects. PU-GCN (Point Upsampling-Graph Convolution Network) later built upon this by replacing the expansion MLP with a graph-convolution upsampling unit called NodeShuffle and paired it with a GCN feature extractor [30]. Recently, PU-Transformer introduced the first transformer-based upsampler to exploit both local and long-range relations [31]. While these methods deliver state-of-the-art results on synthetic shapes and man-made objects, their behavior on the irregular geometry of vegetation—and in LiDAR-derived point clouds more broadly—remains largely untested.

1.1.2. Upsampling in Vegetated Landscapes

Upsampling LiDAR data from forests and other natural vegetation introduces unique challenges. In natural vegetation, aerial LiDAR point clouds exhibit uneven densities—upper vegetation layers are well sampled due to their proximity to the sensor, while lower vegetation layers and the ground experience significantly reduced returns, introducing complexity that differs from man-made environments. Zhang and Filin [32] highlighted that most existing research had focused on upsampling point clouds of man-made objects, with little attention to natural scenes. They found that standard 3D interpolation or naïve point densification often leads to over-smoothed results in forests, since such methods ignore fine local variations in structure. To address this, Zhang and Filin proposed a graph convolutional network with a global attention mechanism that exploits vegetation’s self-similar geometric patterns for superior vegetated landscape upsampling. Nevertheless, that work relied solely on the geometric information in the LiDAR point cloud, without incorporating external imagery or multi-modal data.

1.2. Utilizing Cross-Attention for Multi-Modal Fusion

Cross-attention mechanisms have proven valuable for multi-modal data fusion in remote sensing, though their application has largely centered on integrating various 2D raster datasets [33,34,35,36]. In remote sensing, the primary method for fusing 3D LiDAR data with imagery involves rasterizing the LiDAR information, most often by integrating digital surface models (DSMs) with hyperspectral data [37,38,39]. Consequently, the direct fusion of individual LiDAR point features with imagery using cross-attention represents a largely unexplored area in remote sensing research. Conversely, the broader computer vision community actively develops and utilizes such direct point-to-image cross-attention techniques for enhancing detailed 3D scene perception [40,41,42].

1.2.1. Attention-Based Multi-Modal Upsampling

Building on these advances, we introduce an upsampling model that leverages attention mechanisms to capture both local and global context while fusing LiDAR with optical and radar inputs. Transformer-based architectures have recently shown promise in 3D point cloud tasks by modeling long-range dependencies in point sets. Our network adopts a Local-Global Point Attention block structure inspired by this paradigm. At the local scale, a multi-head variant of a point transformer architecture developed by Zhao et al. [43] applies self-attention within each point’s neighborhood to learn fine-grained spatial details. This “multi-head” approach enables the model to learn multiple, distinct feature representations in parallel; for instance, one head may learn to model fine-scale canopy texture while another captures broader branch-level geometry. At the global scale, we incorporate a position-aware multi-head attention mechanism over the entire point cloud to ensure structural coherence. To maintain computational efficiency, we implement this global attention with a FlashAttention [44] algorithm, allowing exact multi-head attention across thousands of points in a memory-efficient manner. By combining local and global attention pathways, the model preserves small-scale features (e.g., individual tree crown shapes) while enforcing consistency in larger-scale patterns (e.g., stand-level canopy height gradients). This architecture extends prior point upsampling networks but is uniquely tailored to handle multi-modal inputs and the complexities of natural scenes.

The primary scientific contribution of our study is not merely a new network architecture, but rather the exploration of a fused-modality approach to LiDAR point cloud upsampling. In contrast to previous methods that input only sparse LiDAR points, we evaluate how incorporating additional imagery (optical NAIP and L-band SAR) can improve the reconstruction of vegetation structure. We also explicitly examine the temporal dimension by testing models in areas with known canopy growth or loss since the original LiDAR acquisition, an aspect largely overlooked in prior research.

1.3. Research Questions

• RQ1: To what extent does incorporating individual imagery modalities, (a) high-resolution optical imagery or (b) L-band Synthetic Aperture Radar (SAR) imagery, lower the point-cloud reconstruction error (measured by the Chamfer distance) compared to a baseline upsampling model that uses only the initial sparse LiDAR as input?

  -

  Hypothesis: Both modalities will reduce reconstruction error, but optical imagery will yield superior results.

  -

  Reasoning: The finer ground sampling distance of optical imagery provides high-resolution texture essential for fine-scale detail. While L-band SAR is sensitive to volumetric structure, its coarser resolution is a limitation.

• RQ2: Does simultaneously fusing high-resolution optical and L-band SAR imagery yield additional reconstruction accuracy gains beyond the best single-modality model, indicating complementary rather than redundant information?

  -

  Hypothesis: The fused optical and SAR model will achieve the lowest reconstruction error, outperforming both single-sensor models.

  -

  Reasoning: Each sensor captures a different aspect of vegetation structure. The model’s attention-based fusion is expected to leverage optical texture to define canopy boundaries and SAR backscatter to reconstruct internal volume.

• RQ3: How does reconstruction error change with net canopy-height gains and losses since the initial airborne LiDAR survey, and do the optical, SAR, and fused models mitigate these errors more effectively than a baseline upsampling model that uses only the initial sparse LiDAR as input?

  -

  Hypothesis: Errors will scale with canopy change and be greater for losses than gains. Model performance will stratify accordingly: the fused model will best mitigate these errors, followed by single-modality models, with the baseline performing poorest, though it may capture some uniform growth.

  -

  Reasoning: The predicted error asymmetry stems from the different nature of vegetation dynamics. Growth is often an incremental extrapolation of existing structure, whereas disturbance-driven loss (e.g., fire, treefall) can be abrupt and total. This creates a complete information void for removed canopy in the legacy LiDAR, a more significant reconstruction challenge than modeling gradual growth.

2. Materials and Methods

The core challenge addressed by this research is the enhancement of sparse and outdated national airborne LiDAR datasets. To train and validate supervised upsampling models, we collected dense UAV LiDAR as a benchmark of actual vegetation structure. The sections below describe the study areas where this data was collected, the multi-modal input data sources, and steps taken to ensure spatial alignment and consistency across all modalities.

2.1. Study Area

The study area (Figure 2) consists of two separate sites in Southern California, USA. The first study area includes parts of the 24-square-kilometer Sedgwick Reserve, managed by the University of California, Santa Barbara, and the adjacent 11.5 square kilometer Midland School property. Both sit within the San Rafael Mountain foothills of Santa Barbara County (Figure 3).

The area spans elevations from 287 to 852 meters and supports a mosaic of vegetation types with varying canopy architectures, including coastal sage scrub, native grasslands, chaparral (shrublands), coast live and blue oak woodlands, valley oak savanna, riparian habitats, and gray pine forests. UAV LiDAR was collected on four sites (totaling about 71 hectares) within the study area (Table 1). Given their similar terrain and vegetation, two sites were used for training, while the two smallest were withheld for independent model evaluation.

The second study area (Figure 4) comprises 197 hectares within and adjacent to the Volcan Mountain Wilderness Preserve in the Peninsular Range of Southern California. The reserve is managed by San Diego County and the Volcan Mountain Foundation and ranges in elevation from about 1220 meters to over 1675 meters. It hosts diverse plant communities, including oak woodlands, chaparral, mixed conifer forests, and grasslands. To ensure robust model evaluation across this ecological gradient, roughly 30 percent of the area (58 hectares) was reserved for testing and validation (Table 1). The three holdout zones (Figure 4) used for model evaluation were selected to reflect the site’s vegetation diversity: the northernmost area includes chaparral that replaced forest following wildfires in 2003 and 2005; the central zone contains dense mixed-conifer and riparian vegetation interspersed with oak woodlands and chaparral; and the southernmost zone is predominantly semi-continuous oak canopy.

2.2. The Data

All remote-sensing assets were co-registered within a common tiling framework covering both study sites. First, the UAV-LiDAR acquisition footprints were tessellated into 10 m × 10 m analysis tiles with 15% overlap, yielding an initial set of 9,800 tiles at Sedgwick–Midland and 26,557 tiles at Volcan Mountain. Each tile served as the spatial key for assembling a four-layer data stack (Table 2)

By standardizing on overlapping 10 m patches, we guarantee that each training example draws from the same footprint across sensors—ensuring the model learns consistent, co-registered features from LiDAR, SAR, and imagery.

2.2.1. UAV LiDAR Data

Sedgwick and Midland Sites - Between June and October 2023, UAV LiDAR data were collected by San Diego State Geography Department Staff on the Sedgwick and Midland School Sites using a DJI Matrice 300 drone equipped with a TrueView 515 LiDAR. The drone was flown at an altitude of approximately 60 meters above ground level, achieving a point density of around 300 points per square meter.

Volcan Mountain Site - On October 25, 2024, the same Geography Department staff conducted flights using a DJI Matrice 350 drone equipped with a TrueView 540 LIDAR system. This newer drone and sensor was flown at a higher altitude of approximately 110 meters above ground level, and still achieved a point density of over 600 points per square meter.

2.2.2. Crewed Airborne LiDAR

The available crewed airborne LiDAR (C-ALS) data includes two separate 3DEP datasets. The Sedgwick 3DEP dataset, collected in 2018, has a point density of 22 pts/m². The C-ALS data for the Volcan Mountain site was collected between October 2015 and November 2016 and has a point density of 6.3 pts/m² [45]. Both datasets were obtained from Microsoft’s Planetary Computer [46,47]

2.2.3. Synthetic Aperture Radar (SAR)

Fully polarimetric L-band imagery (23.84 cm wavelength) from NASA’s UAVSAR system was obtained with the Alaska Satellite Facility’s asf_search Python API [48]. The UAVSAR flights were conducted from a Gulfstream-III platform with bidirectional acquisitions from opposite look directions at an average altitude of 12,495 meters, providing multi-perspective radar coverage of the landscape at 6.17-meter ground resolution. The specific campaigns and acquisition details for each study site are summarized in Table 3. For every 10 m × 10 m 3DEP tile we extracted a co-centered 20 m × 20 m UAVSAR chip to accommodate layover and shadow extent, then bilinearly resampled each chip to 5 m GSD before fusion with NAIP imagery and LiDAR.

2.2.4. High-Resolution Aerial Imagery

We ingested NAIP imagery through the Microsoft Planetary Computer STAC API [47] for survey years 2014–2022. NAIP provides four-band (Red, Green, Blue, Near-Infrared) orthoimagery of the conterminous United States, collected at peak green-up on a two- to three-year cycle. Prior to the 2016 flight season, data were delivered at 1 m ground-sample distance (GSD); since 2016 the native resolution has been 60 cm. A complete timeline of these acquisitions and their resolutions is provided in Table 3. For every 10 m × 10 m 3DEP tile we extracted a 20 m × 20 m NAIP chip centered on the same point to accommodate viewing-geometry variance and to capture neighboring shadows. All NAIP scenes were then resampled to a common 50 cm grid.

2.3. Data Cleaning and Preprocessing

To reduce computational load and give the upsampling network a uniform-density target, we downsampled the UAV LiDAR for every tile with an adaptive anisotropic voxel grid. Each cloud was first voxelized at a 4 cm × 4 cm × 2 cm resolution; if more than 50,000 points remained, the horizontal voxel edge was incrementally enlarged (keeping the vertical edge at 50% of that value) and the filter reapplied until the count fell below the limit. The resulting point sets preserve fine vertical structure while standardizing horizontal density. The dataset was partitioned into training, validation, and test sets using reference polygons to ensure the holdout sets captured the full environmental gradients found in the training data. We enforced quality constraints by requiring a UAV-to-3DEP point ratio above 2.0, and a minimum of 16,000 UAV LiDAR points and 200 3DEP points per tile.

2.4. Data Augmentation

To increase the model’s robustness and prevent overfitting, we expanded the training dataset from 24,000 to 40,000 tiles via data augmentation[49,50]. First, we preferentially selected source tiles for this process, prioritizing those with high structural complexity (z-variance) and large vegetation changes. Each selected source tile was then used to generate a new, augmented sample by applying a random combination of transformations. These included geometric operations (rotations and reflections) applied to all data layers, and point cloud-specific perturbations (random point removal and jittering) applied only to the input LiDAR data.

2.5. Model Architecture

Our multimodal upsampling framework transforms a sparse 3DEP point cloud, plus co-registered NAIP and UAVSAR image chips, into a denser 3-D point cloud (Figure 5). The network is built around the Local–Global Point Attention Block (LG-PAB; §2.6), which provides permutation-invariant feature learning, optional feature-guided upsampling, and long-range geometric context. Five macro–components are arranged in a feed-forward sequence: (1) point feature extraction, (2) imagery encoding, (3) cross-attention fusion, (4) feature expansion and refinement, and (5) point decoding.

Notation Conventions

Throughout this section, we use the following notation for tensor dimensions:

• Counts: $N_{\mathrm{pts}}$ (number of input points), $N_{\mathrm{curr}\_\mathrm{pts}}$ (current points in LG-PAB), $N_{\mathrm{pts}\_\mathrm{up}}=R_{\mathrm{up}}·N_{\mathrm{pts}}$ (upsampled points), $N_{\mathrm{patch}}$ (number of image patches, default 16), $N_{\mathrm{time}}$ (temporal stack length), $N_{\mathrm{looks}}$ (maximum look angles, $\le 2$).
• Dimensions: $D_{\mathrm{coord}}=3$ (coordinate dimension), $D_{\mathrm{attr}}$ (attribute dimension), $D_{\mathrm{p}\_\mathrm{feat}}=256$ (point feature dimension), $D_{\mathrm{p}\_\mathrm{in}}$ (input point feature dimension), $D_{\mathrm{p}\_\mathrm{out}}$ (output point feature dimension), $D_{\mathrm{token}}=128$ (token dimension).
• Channels: $C_{\mathrm{naip}}=4$ (NAIP channels), $C_{\mathrm{uavsar}}=6$ (UAVSAR channels).
• Image dimensions: $H_{\mathrm{naip}}=W_{\mathrm{naip}}=40$ (NAIP height/width), $H_{\mathrm{uavsar}\_\mathrm{pr}}=W_{\mathrm{uavsar}\_\mathrm{pr}}=4$ (UAVSAR patch region height/width).
• Other: $R_{\mathrm{up}}$ (upsampling ratio, default 2), $k_{\mathrm{knn}}=16$ (k-nearest neighbors).

For tensors, we use the notation TensorSymbol: (dim₁, dim₂, ..., dim_k) to describe their dimensions.

Given an input cloud $P_{\mathrm{in}}:(N_{\mathrm{pts}},D_{\mathrm{coord}})$ with points ${\left\{{\mathbf{x}}_i\right\}}_{i=1}^{N_{\mathrm{pts}}}\subset {\mathbb{R}}^{D_{\mathrm{coord}}}$ and attributes ${\mathbf{a}}_i:\left(D_{\mathrm{attr}}\right)$, the network outputs $P_{\mathrm{out}}:(N_{\mathrm{pts}\_\mathrm{up}},D_{\mathrm{coord}})$ with points ${\left\{{\widehat{\mathbf{x}}}_j\right\}}_{j=1}^{N_{\mathrm{pts}\_\mathrm{up}}}$ where $N_{\mathrm{pts}\_\mathrm{up}}=R_{\mathrm{up}}·N_{\mathrm{pts}}$ (typically $R_{\mathrm{up}}=2$).

An overview is:

• LG-PAB Extractor${\mathcal{E}}_{\mathrm{pt}}$$\to$ local–global point features $F:(N_{\mathrm{pts}},D_{\mathrm{p}\_\mathrm{feat}})$;
• Imagery Encoders${\mathcal{E}}_{\mathrm{opt}},{\mathcal{E}}_{\mathrm{rad}}$$\to$ NAIP and UAVSAR patch embeddings $E_{\mathrm{opt}},E_{\mathrm{rad}}:(N_{\mathrm{patch}},D_{\mathrm{token}})$;
• Cross-Attention Fusion${\mathcal{F}}_{\mathrm{ca}}$$\to$ enriched point features $F_{\mathrm{fused}}:(N_{\mathrm{pts}},D_{\mathrm{p}\_\mathrm{feat}})$;
• LG-PAB Expansion & Refinement$\to$ upsampled features $F^{\uparrow }:(N_{\mathrm{pts}\_\mathrm{up}},D_{\mathrm{p}\_\mathrm{feat}})$ and coordinates $P^{\uparrow }:(N_{\mathrm{pts}\_\mathrm{up}},D_{\mathrm{coord}})$;
• MLP Decoder$\to$ residual offsets $\Delta P:(N_{\mathrm{pts}\_\mathrm{up}},D_{\mathrm{coord}})$ and final coordinates $P_{\mathrm{out}}:(N_{\mathrm{pts}\_\mathrm{up}},D_{\mathrm{coord}})$.

2.6. The Local–Global Point Attention Block (LG-PAB)

Figure 6 presents a flow-chart of the Local–Global Point Attention Block, the fundamental unit used three times in our architecture (extraction, expansion, and refinement stages). Each LG-PAB converts an input tuple $\langle X,P\rangle$ consisting of point features $X:(N_{\mathrm{curr}\_\mathrm{pts}},D_{\mathrm{p}\_\mathrm{in}})$ and 3-D coordinates $P:(N_{\mathrm{curr}\_\mathrm{pts}},D_{\mathrm{coord}})$ into refined (and optionally upsampled) features and positions. The block proceeds through the stages that appear in the diagram:

• Local Attention Block A Multi-Head Point Transformer operates on a $k_{\mathrm{knn}}$-nearest-neighbour graph ($k_{\mathrm{knn}}=16$) to capture fine-scale geometry. Its output passes through a two-layer Feed-Forward Network (FFN) with GELU activation, producing an intermediate tensor $Z:(N_{\mathrm{curr}\_\mathrm{pts}},R_{\mathrm{up}}·D_{\mathrm{p}\_\mathrm{out}})$. When the upsampling ratio $R_{\mathrm{up}}=1$ this step already delivers the final per-point features.
• Feature-Guided Upsampling (optional) If $R_{\mathrm{up}}>1$ (expansion stage) the intermediate tensor is reshaped to $[N_{\mathrm{curr}\_\mathrm{pts}},R_{\mathrm{up}},D_{\mathrm{p}\_\mathrm{out}}]$, effectively cloning each feature vector $R_{\mathrm{up}}$ times. A small Position-Generator MLP then predicts a 3-D offset for every clone, yielding new coordinates $\widehat{P}=P^{\mathrm{rep}}+\Delta P:(N_{\mathrm{curr}\_\mathrm{pts}}·R_{\mathrm{up}},D_{\mathrm{coord}})$. The features are flattened back to $[N_{\mathrm{curr}\_\mathrm{pts}}·R_{\mathrm{up}},D_{\mathrm{p}\_\mathrm{out}}]$.
• Global Attention Block To impose long-range coherence, the upsampled (or original) features are processed by Position-Aware Global Flash Attention. Coordinates are first embedded by a two-layer MLP, concatenated to the features, and fed to a four-head FlashAttention layer that attends across all points in the tile. A second FFN refines the attended features, after which residual connections and LayerNorm complete the block.

2.7. Imagery Encoders

Optical (NAIP) and radar (UAVSAR) image chips are processed by a shared five-stage encoder (with modality specific weights) that converts each image stack into a fixed set of patch tokens. The encoder stages—illustrated in Figure 7—are:

• Patch Tokenization. A two-layer Conv–GELU–Conv stem with stride-1 followed by average pooling (stride=$\mathtt{patch}\_\mathtt{size}$) extracts features on a $4\times 4$ grid. This design mirrors Shifted Patch Tokenization, which adds local texture bias to Vision Transformers (ViTs) and improves sample efficiency on small datasets [51]. The $N_{\mathrm{patch}}=16$ patches are flattened to $Z_{\mathrm{tok}}:(N_{\mathrm{patch}},D_{\mathrm{token}})$ and normalized via LayerNorm.
• 2-D Positional Encoding. Normalized patch centers are embedded via a two-layer MLP and added to the tokens: $Z_{\mathrm{pos}}=Z_{\mathrm{tok}}+{\mathrm{MLP}}_{\mathrm{pos}}(x,y)$.
• Transformer Encoder Block. A 4-head self-attention layer is paired with LayerScale ($\gamma \approx {10}^{-5}$), a learnable scalar that stabilizes deep transformers [52]. The accompanying MLP includes a depth-wise 1-D convolution, following the ConViT approach to introduce a soft convolutional inductive bias [53]. The block outputs $Z_{\mathrm{enc}}:(N_{\mathrm{patch}},D_{\mathrm{token}})$.
• Temporal GRU–Attention Head. For inputs with temporal length $N_{\mathrm{time}}$, a bidirectional Gated Recurrent Unit (GRU) with attention pooling aggregates tokens into patch descriptors $E_{\mathrm{patch}}:(N_{\mathrm{patch}},D_{\mathrm{token}})$.

UAVSAR Look Angle Mean Pooling

• UAVSAR. Input $I_{\mathrm{uavsar}}:(N_{\mathrm{time}},N_{\mathrm{looks}},C_{\mathrm{uavsar}},H_{\mathrm{uavsar}\_\mathrm{pr}},W_{\mathrm{uavsar}\_\mathrm{pr}})$ with $N_{\mathrm{looks}}\le 2$. After stage 3, we average features across available look angles to produce $Z_{\mathrm{avg}}:(N_{\mathrm{time}},N_{\mathrm{patch}},D_{\mathrm{token}})$ before temporal processing. Otherwise, the pipeline matches NAIP.

Both encoders output token matrices $E_{\mathrm{opt}}$ and $E_{\mathrm{rad}}:(N_{\mathrm{patch}},D_{\mathrm{token}})$ that are normalized and concatenated before fusion.

2.8. End-to-End Upsampling Pipeline

1) Local–Global Feature Extraction.

The sparse 3DEP cloud—concatenated with intensity, return number, and number or returns (6 attributes total)—is processed by the first Local–Global Point Attention Block (§Section 2.6). The output is a set of point features $F:(N_{\mathrm{pts}},D_{\mathrm{p}\_\mathrm{feat}})$ that encode both neighborhood morphology and tile-level context.

2) Imagery Encoders.

Co-registered NAIP and UAVSAR chips are independently tokenized and fed to lightweight Transformer encoders ($N_{\mathrm{patch}}$ patches, embed dim $D_{\mathrm{token}}$). The optical encoder (RGB+NIR) outputs patch embeddings $E_{\mathrm{opt}}:(N_{\mathrm{patch}},D_{\mathrm{token}})$, whereas the radar encoder (L-band polarimetry) outputs $E_{\mathrm{rad}}:(N_{\mathrm{patch}},D_{\mathrm{token}})$. When multiple acquisition dates exist, per-modality tokens are fused temporally via a shared GRU–attention head.

3) Cross-Attention Fusion.

Point features act as queries and image patches as keys/values in a four-head cross-attention block. Scaled dot-product scores are masked for patches whose centroid is more than 8 meters from the query point. The fused representation $F_{\mathrm{fused}}:(N_{\mathrm{pts}},D_{\mathrm{p}\_\mathrm{feat}})$ augments every point with spectral texture and volumetric back-scatter cues, improving discrimination of canopy surfaces versus gaps.

4) Feature-Guided Upsampling.

A second LG-PAB with ratio $R_{\mathrm{up}}=2$ expands $F_{\mathrm{fused}}$ and predicts offsets $\Delta P$ that generate $N_{\mathrm{pts}\_\mathrm{up}}$ candidate coordinates. This stage doubles point density while maintaining local topology.

5) Feature Refinement.

A third LG-PAB (ratio $R_{\mathrm{up}}=1$) re-computes local and global attention on the enlarged cloud, ironing out artifacts introduced during expansion and propagating context across newly formed neighborhoods.

6) Coordinate Decoding.

Finally, a four-layer MLP ($D_{\mathrm{p}\_\mathrm{feat}}\to 128\to 64\to 32\to D_{\mathrm{coord}}$) regresses residual offsets that are added to the upsampled positions, producing the higher-resolution prediction $\widehat{P}:(N_{\mathrm{pts}\_\mathrm{up}},D_{\mathrm{coord}})$.

2.9. Training Protocol and Analysis

To systematically evaluate our research questions, we established a rigorous training and analysis protocol. The full dataset, comprising tiles from both study areas, was partitioned into training, validation, and testing subsets. The training set consists of 24,000 original tiles, which was expanded to 40,000 through the data augmentation process described previously. A separate set of 3,792 tiles was reserved for validation during training, and a final hold-out set of 5,688 tiles was used for testing and performance evaluation (Table 4).

To isolate the impact of each data modality, we trained four distinct model variants (Table 5). A baseline model was trained using only the sparse 3DEP LiDAR input. Two single-modality models were trained by fusing the LiDAR with either NAIP optical imagery or UAVSAR radar data. Finally, a fully fused model was trained using all three data sources simultaneously.

All model variants were trained using an identical architecture, set of hyperparameters, and training protocol to ensure a fair comparison. Key model configuration parameters, including feature dimensions and attention head counts, are provided in Table 6. The models were trained for 100 epochs on four NVIDIA L40 GPUs using the ScheduleFreeAdamW optimizer [54] and a density-aware Chamfer distance loss function (Eq. 9 in [55]). Further details on the training protocol and hardware are listed in Table 7.

We evaluated our research questions using non-parametric statistical tests to account for non-normality in the error distribution. For RQ1 and RQ2, we used Wilcoxon signed-rank tests to compare reconstruction error (measured by Chamfer distance) between models, with median percentage change and rank-biserial correlation as effect size measures. For RQ3, we used Spearman rank correlations to analyze the relationship between reconstruction error and absolute canopy height change across all models. We further split the dataset into canopy gains (N=2423) and losses (N=3264) to examine potential asymmetries in error patterns. Fisher r-to-z transformations were used to statistically compare correlation coefficients between models. All significance values are reported at $\alpha$=0.05, with bold values indicating statistically significant results.

3. Results

A summary of the reconstruction performance, measured by Chamfer distance (CD), across all models is presented in Table 8. The 3DEP LiDAR-only baseline model provided a substantial improvement over using the raw, sparse input data, reducing the median error by over 60% and tightening the error distribution (Figure 8 and Figure 9). When comparing the performance across all models (Figure 10), the fused model consistently achieved the lowest median error and smallest interquartile range, indicating the most robust performance. To address our first research question (RQ1) on the impact of individual modalities, we compared the error distributions of the single-modality models against the baseline (Table 9).

Both high-resolution optical imagery (NAIP) and L-band SAR imagery significantly reduced reconstruction error compared to the LiDAR-only baseline, with optical imagery providing slightly larger improvements (0.5% vs 0.3%). While statistically significant, the modest effect sizes (0.088 and 0.062 respectively) suggests that single-modality improvements over the baseline upsampling approach are limited in magnitude or may be concentrated in a limited number of tiles. Next, to evaluate our second research question (RQ2), we assessed whether fusing both imagery types yielded additional benefits over the best single-modality model (Table 9).

The fusion of both optical and SAR imagery yielded additional reconstruction accuracy gains (0.7% median reduction) beyond using NAIP alone, with a stronger effect size (0.133) than either individual modality achieved in RQ1. This statistically significant improvement supports our hypothesis that the two modalities contain complementary information that can be effectively combined through attention-based fusion. The results demonstrate that multi-modality approaches can leverage different sensing capabilities to achieve superior point cloud reconstruction (Figure 11).

Finally, to evaluate our third research question (RQ3) on the impact of vegetation change, we correlated model error with the magnitude of canopy height changes between the legacy 3DEP and recent UAV LiDAR surveys. Table 10 shows the overall Spearman rank correlations. To investigate this relationship further, we split the dataset into areas of net canopy gain and net canopy loss, with the results of this extended analysis presented in Table 11.

All models showed strong correlations with absolute canopy height change, confirming that reconstruction error systematically increases with vegetation structure changes since the original LiDAR collection. The baseline model exhibited the strongest correlation with canopy change ($\rho =0.650$), while the fused model showed the weakest ($\rho =0.582$), with this difference being statistically significant ($z=5.868$, $p<0.001$). Importantly, the extended analysis revealed this pattern was driven primarily by canopy losses, where the baseline model performed significantly worse than all other models, particularly the fusion approach ($z=6.074$, $p<0.001$), while for canopy gains, all models performed similarly without statistically significant differences. These findings partially support our hypothesis that advanced models better mitigate error from canopy changes, but specifically for canopy removal scenarios where legacy LiDAR contains no information about the removed vegetation (Figure 12 and Figure 13).

4. Discussion

This study demonstrates the significant potential of attention-based deep learning models to enhance sparse and outdated airborne LiDAR point clouds by fusing them with more frequently acquired optical (NAIP) and Synthetic Aperture Radar (UAVSAR) imagery. Our findings directly address the critical need for up-to-date, high-resolution 3D vegetation structure information in applications like wildfire risk modeling and ecological monitoring.

The core success of our approach lies in the effective fusion of multi-modal data. As hypothesized (RQ1), both NAIP optical imagery and L-band UAVSAR imagery, when individually integrated, improved point cloud reconstruction accuracy compared to a baseline model relying solely on sparse LiDAR. NAIP, with its finer spatial resolution, offered a slightly greater enhancement, likely due to its ability to delineate canopy edges and small gaps with high fidelity. However, the true advancement was observed when these modalities were combined (RQ2). The fused model, leveraging both NAIP’s textural detail and UAVSAR’s structural sensitivity, outperformed single-modality enhancements. This confirms our hypothesis that these sensors provide complementary, rather than redundant, information, and that the cross-attention mechanisms within our architecture can effectively identify and leverage these synergistic relationships. This synergy is particularly valuable for capturing the complex, heterogeneous nature of vegetation.

Our investigation into temporal dynamics (RQ3) revealed that all models, including the baseline, exhibited increased reconstruction error in areas with substantial canopy change since the initial LiDAR survey. This is expected, as the input LiDAR reflects a past state. However, the fused model demonstrated the most robust performance, showing a significantly weaker correlation between error and the magnitude of canopy change, especially in areas of canopy loss. This is a crucial finding: by incorporating more recent imagery, particularly the optical data that clearly depicts vegetation absence, the model can more effectively correct for outdated LiDAR information. The baseline model, lacking this current-state information, struggled most in loss scenarios. Even in areas of canopy growth, while not as pronounced as with loss, the image-informed models offered an advantage by providing cues about new or denser vegetation that the original sparse LiDAR could not capture. This capacity to "update" historical LiDAR datasets significantly enhances their utility for long-term monitoring and management, especially in landscapes prone to rapid changes from disturbances or growth.

The developed Local-Global Point Attention Block (LG-PAB) proved to be a robust architectural component. Its ability to capture both fine-grained local details through neighborhood-level self-attention and broader structural coherence via global attention across the entire point patch is central to its success. This hierarchical attention is well-suited to the fractal-like patterns often observed in natural vegetation.

Despite these promising results, some limitations exist. The improvements, while statistically significant, were modest in terms of percentage change in Chamfer distance. This suggests that while fusion helps, there might be inherent limits to upsampling very sparse LiDAR or that the current metrics may not fully capture all aspects of structural improvement relevant to ecological applications. Furthermore, our study was conducted in Southern California’s Mediterranean ecosystems; model performance and the relative contributions of optical versus radar data might vary in different biomes with distinct vegetation structures and phenology.

Beyond the primary 2x upsampling task, we also conducted a preliminary investigation into the model’s potential for higher-ratio densification (8x). This exploratory model was scaled up significantly, with larger feature dimensions (512 point, 192 image), more local attention heads for extraction and expansion (16), and a total of 125M parameters. With our hardware, this increased model size necessitated reducing the batch size to one per GPU. The results (Figure 14) show that the architecture can indeed produce highly dense outputs that qualitatively approach the reference data. The appendix provides further qualitative examples from both the standard 2x and experimental 8x models, illustrating their performance across different vegetation change scenarios (Figure 15, Figure 16 and Figure 17).

A recommended area for future research involves adapting the validated multi-modal feature extraction pipeline for direct prediction of key vegetation structure rasters, such as Canopy Height Models (CHM), canopy cover, Above-Ground Biomass (AGB), and fuel types. Such an adaptation would entail replacing the current point cloud generation head with task-specific regression or classification heads, potentially broadening the practical applicability of this work. Integrating geometric priors, like Digital Surface Models (DSMs) and Digital Terrain Models (DTMs), into the loss function also represents a valuable direction. This could not only enforce greater structural realism but also enable the calculation of reconstruction error at various canopy strata, offering deeper insights into model performance. Furthermore, fine-tuning emerging foundation-model vision transformers, such as the Clay model [56], as shared encoders for NAIP and UAVSAR imagery, warrants exploration to leverage large-scale pretraining for enhanced feature representation. Complementary investigations could include a thorough evaluation of the UAVSAR encoder, particularly optimizing multi-look fusion beyond simple averaging and assessing resampling impacts, alongside ablation studies on the LG-PAB and imagery encoders to pinpoint key architectural contributions and guide further optimization. Future architectural research could also explore simplifying the Local-Global Point Attention Block into a pure Point Attention Block by replacing the k-NN local attention with a global attention module. For greater scalability, this could be paired with a latent attention strategy [57] to bypass the quadratic complexity inherent to self-attention.

5. Conclusions

This research successfully demonstrates that attention-based deep learning, leveraging our novel Local-Global Point Attention Block, can significantly enhance sparse airborne LiDAR point clouds in vegetated landscapes through the fusion of more recent optical and radar imagery. We have shown that while individual imagery modalities provide benefits, their combined use yields superior reconstruction accuracy, particularly in mitigating errors arising from vegetation changes over time. Specifically, high-resolution optical imagery (NAIP) proved slightly more effective as a standalone ancillary dataset than L-band SAR within our framework, but the fusion of both offered the best performance, validating the complementary nature of these sensors. A key contribution is the model’s ability to substantially reduce reconstruction degradation in areas of vegetation loss, thereby increasing the utility of historical LiDAR archives. This study presents a novel approach to direct 3D point cloud upsampling using multi-modal fusion in complex natural environments, moving beyond prevalent raster-based enhancement techniques and paving the way for more accurate and timely assessments of vegetation structure.