Spatial-Aware Uncertainty Quantification for Low-Cost PM₂.₅ Sensors: GeoConformal Prediction vs. Monte Carlo Dropout Under Wildfire Distribution Shift

1. Introduction

Air pollution, exacerbated by climate-driven events like wildfires, poses significant risks to both air quality and human health [1]. Fine particulate matter (PM₂.₅) is of particular concern because it can penetrate deep into the respiratory system and is strongly associated with adverse health outcomes, underscoring the need for accurate monitoring [2]. Regulatory-grade sensors such as those from the United States Environmental Protection Agency (EPA) provide high-quality measurements but are limited in number and spatial coverage. As a result, they often fail to capture localized pollution variability, especially during extreme events [3]. This limitation becomes particularly important during wildfire events, where PM₂.₅ concentrations can vary substantially across space, even within the same region [4]. As a result, there is a growing need for monitoring approaches that can capture localized air quality dynamics beyond the coverage of existing regulatory networks.

Low-cost sensors (LCS) have emerged as a promising supplementary technology for regulatory monitoring networks owing to their affordability, accessibility, and small size, enabling denser spatial coverage [5]. Expanding low-cost sensor networks can help preserve air quality monitoring coverage when individual sensors fail or are destroyed during extreme events [6]. Among the most widely deployed LCS networks, PurpleAir operates tens of thousands of community-based sensors globally, offering near-real-time PM₂.₅ data at high spatiotemporal resolution [7]. While PurpleAir's real-time sensor readings are publicly viewable, bulk and historical data access through their API requires a paid subscription, limiting its utility for large-scale retrospective research [2]. Clarity, another LCS network, provides community-scale PM₂.₅ monitoring with publicly accessible data, making it a viable alternative for research applications [8]. Raw measurements from these LCS require calibration to correct systematic bias relative to regulatory-grade monitors [9]. This calibration is typically performed using statistical or machine learning models that learn relationships between sensor readings, meteorological variables, and reference observations [10]. Once calibrated, these models are used to generate spatial predictions or interpolations of PM₂.₅ concentrations, particularly in areas lacking dense monitoring coverage [11].

Despite improvements in calibration accuracy, most studies evaluate sensor performance using metrics such as mean absolute error (MAE), root mean square error (RMSE), or coefficient of determination (R²). These metrics quantify average error but do not assess prediction reliability or confidence under different conditions [5,12]. This limitation is especially critical during wildfire events, which can introduce substantial distribution shifts in PM₂.₅ concentrations relative to typical training conditions [7]. To address this gap, several model-based uncertainty quantification (UQ) methods, including Bayesian neural networks (BNNs), Monte Carlo Dropout (MCD), and deep ensembles (DE), have been developed to quantify uncertainties by modeling variability in model outputs [13]. These UQ methods remain dependent on model assumptions and the representativeness of the training distribution [14]. Under a wildfire-induced distribution shift, model-derived uncertainty may become miscalibrated, leading to overconfident or unreliable prediction intervals [15]. On the other hand, conformal prediction has emerged as a distribution-free alternative for UQ that constructs prediction intervals from observed residual errors, providing distribution-free coverage guarantees [16,17]. GeoConformal prediction (GCP) extends this framework to spatial settings by weighting calibration errors according to geographic proximity, producing location-dependent prediction intervals with formal coverage guarantees [14].

While interest in predictive uncertainty for air quality estimation has grown, a critical gap remains in the systematic evaluation of the reliability of predictive uncertainty under conditions of spatial sparsity and wildfire-induced distribution shift. This study addresses this gap by systematically evaluating GCP and MCD reliability for low-cost PM₂.₅ sensor calibration across varying sensor separations and wildfire conditions, establishing a reliability-centered framework for UQ. The research objectives are listed below:

• Develop a transformer-based model for PM₂.₅ calibration and quantify uncertainty using GCP and MCD to produce reliable, well-calibrated estimates with uncertainty bounds across two study areas, California (CA) and the Northeast (NE) United States.
• Analyze how calibration performance and UQ reliability vary with increasing spatial distance between collocated sensor pairs (1 km, 5 km, and 10 km thresholds) across both study areas, to understand the spatial limitations of LCS calibration.
• Evaluate and compare the reliability of GCP and MCD under wildfire and non-wildfire conditions, focusing on empirical coverage and interval width to assess how wildfire-induced distribution shift affects each uncertainty method.

The remainder of this paper is organized as follows. Section 2 reviews prior research on UQ for low-cost PM₂.₅ sensor calibration, including performance under wildfire conditions. Section 3 introduces the study area and datasets used in this work. Section 4 presents the calibration modeling framework, experimental design, and evaluation metrics. Section 5 presents the UQ results and reliability analysis, followed by a discussion in Section 6 and conclusions in Section 7.

2. Literature Review

This section reviews the literature relevant to the three core dimensions of this study. Section 2.1 examines recent advances in low-cost PM₂.₅ sensor calibration, including machine-learning and deep-learning methods that form the basis of the calibration framework used in this work. Section 2.2 discusses the challenges of air quality monitoring during wildfire events, in which extreme PM₂.₅ concentrations can degrade both calibration accuracy and uncertainty estimates. Section 2.3 surveys existing UQ methods for LCS calibration, identifying the gap between model-based approaches, such as MCD, and distribution-free alternatives, such as GCP, that this study aims to address.

2.1. Low-Cost Air Quality Sensor Calibration

Recent research on low-cost PM₂.₅ sensor calibration has increasingly emphasized the importance of field calibration under real-world deployment conditions, as raw measurements from LCS often exhibit systematic bias relative to regulatory-grade sensors. One study focusing on field-based calibration demonstrated that accounting for PM₂.₅ levels and temperature can reduce bias and improve agreement with reference monitors [18]. Specifically, LCS are highly sensitive to environmental factors such as temperature, humidity, and pollutant composition, and calibration approaches that do not account for this variability may not generalize well when applied across diverse settings, highlighting the risk of deploying sensors without local calibration [19,20].

Expanding on this, recent work applies machine-learning-based calibration methods to improve agreement between LCS and reference-grade monitors [21]. Multiple studies demonstrate that machine learning calibration can significantly reduce systematic error and improve model performance relative to raw sensor measurements [22,23,24]. Traditional machine learning models such as linear regression, random forest, and gradient boosting, provide strong baseline calibration performance [10,25,26]. More recently, deep learning approaches show superior results, with recurrent neural network (RNN) models such as Long Short-Term Memory (LSTM) capturing temporal dependencies in sensor data and outperforming traditional machine learning approaches for PM₂.₅ calibration [12]. A systematic evaluation of machine learning models for LCS calibration further confirms that LSTM outperforms traditional approaches [2]. Despite these advances, most deep learning calibration studies do not evaluate predictive uncertainty, limiting the ability to assess the reliability of individual estimates, a gap that motivates the formal UQ methods examined in Section 2.3.

2.2. Low-Cost Air Quality Sensing During Wildfires

Wildfires are an increasingly common source of extreme PM₂.₅ exposure, yet regulatory monitoring networks often lack the spatial coverage needed to capture localized smoke impacts [27]. Studies demonstrated that networks of LCS can fill critical spatial gaps, enabling detailed monitoring of rapid PM₂.₅ increases across indoor, outdoor, and personal exposure settings [3]. When appropriate device-specific adjustment factors are applied, LCS achieve mean absolute errors below 10 µg/m3 and substantially improve the accuracy of wildfire smoke detection [28,29]. Integrating LCS with regulatory monitoring networks also improves area-wide concentration estimation during wildfire events and reduces interpolation error by over 20% [30]. These findings support the use of LCS for capturing local PM₂.₅ concentrations during wildfire events, especially in regions lacking coverage from regulatory monitors.

However, wildfire events can introduce a significant dataset shift because PM₂.₅ concentrations and environmental conditions during smoke episodes may differ substantially from those represented in the model training data [7]. Such shifts can reduce the generalizability of calibration models and lead to degraded performance, as distributional differences between training and deployment data are a recognized challenge in geospatial machine learning applications [31]. Furthermore, uncertainty estimates that appear well calibrated under in-distribution conditions may become unreliable under dataset shift [15]. While previous studies have primarily focused on improving calibration accuracy during wildfire events, limited attention has been given to systematically evaluating the reliability of calibrated PM₂.₅ estimates and their associated uncertainty estimates under these conditions.

2.3. Uncertainty in LCS Calibration

In this section, two forms of spatial shift in PM₂.₅ calibration are reviewed: variation in PM₂.₅ characteristics and calibration performance across geographic regions and increasing distance between LCS and reference monitors. Firstly, calibration performance can vary across geographic regions due to differences in aerosol composition, meteorology, emissions, and environmental conditions [32]. These regional differences can reduce model transferability and lead to degraded calibration accuracy when models are applied outside the conditions represented in the training data [33]. To address this, a recent work has advocated for region-specific calibration and validation strategies that account for local aerosol and meteorological conditions [34]. Secondly, fine-scale spatial variability in PM₂.₅ measurements can occur even between nearby sensors, reflecting local pollution sources and heterogeneity in observed concentrations [35]. This spatial heterogeneity results in reduced agreement between low-cost and regulatory PM₂.₅ sensors with increasing distance from reference monitors [36,37]. Zhivkov and Fidanova [38] showed that the calibration uncertainty increases with distance from reference monitors, leading to reduced reliability of measurements at greater distances. However, their quantified uncertainty as a distance-dependent empirical growth rate but does not provide individual prediction-level uncertainty or adapt to changing environmental conditions.

Together, these findings highlight that while LCS calibration has advanced considerably through various approaches, most studies have focused on predictive accuracy and have lacked systematic uncertainty quantification. While our previous work systematically compared model-based UQ methods such as BNNs, MCD, and DE for geospatial air quality monitoring [39], it did not evaluate spatially adaptive, distribution-free UQ methods for calibrated PM₂.₅ estimates. This study addresses these gaps by systematically evaluating both model-based and model-agnostic UQ methods for transformer-based PM₂.₅ calibration. A major contribution of this study is the evaluation of GCP as a spatially adaptive, distribution-free UQ framework that generates geographically informed prediction intervals with formal coverage guarantees. In addition, this study systematically examines how uncertainty reliability varies across increasing collocation distances between low-cost and reference sensors and under wildfire-induced distribution shifts.

3. Study Area and Data

3.1. Study Area

This study focuses on two geographically distinct study areas, California and the Northeast United States. These two regions were selected because they represent contrasting aerosol compositions, regulatory monitoring densities, and wildfire exposure regimes, enabling evaluation of model generalizability across climatically and geographically diverse conditions. Figure 1 and Figure 2 depict the locations of AirNow and Clarity sensors across both study areas. In California, 147 AirNow monitors and 1,022 Clarity sensors were used, concentrated in the Los Angeles and San Francisco metropolitan areas. In the Northeast United States, 157 AirNow monitors and 101 Clarity sensors were used. AirNow monitors provide broad spatial coverage across both study areas, while Clarity sensors offer dense coverage over urban regions, which is critical for capturing localized PM₂.₅ variability and supporting high-resolution calibration during extreme events.

Two wildfire events were used as case studies to evaluate model performance under extreme PM₂.₅ conditions. In California, the Palisades Fire and Eaton Fire ignited on January 7, 2025, becoming among the most destructive wildfires in California history [40]. The Palisades Fire burned approximately 23,448 acres in the Pacific Palisades and Malibu areas, while the Eaton Fire burned approximately 14,021 acres near Altadena and Pasadena, collectively causing severe PM₂.₅ degradation across the Los Angeles basin [41]. In the Northeast United States, the Jennings Creek Wildfire ignited in November 2024 along the border of New York and New Jersey, burning approximately 5,000 acres, prompting health advisories for unhealthy air quality across the region [42]. Together, these wildfire events provide distinct and complementary case studies for evaluating the reliability of UQ methods under wildfire-induced distribution shift across both study areas.

3.2. Data Acquisition

3.2.1. AirNow

Hourly ground-level PM₂.₅ concentrations from 2022–2025 were obtained from the U.S. EPA AirNow network via the AirNow API [43]. AirNow is a national air quality reporting partnership that aggregates monitoring data from federal, state, tribal, and local agencies across the United States [44]. Monitoring agencies submit observations from regulatory-grade air quality monitors to AirNow's centralized system, which provides standardized national reporting and near-real-time quality control. These observations served as reference measurements for sensor calibration and uncertainty evaluation.

3.2.2. Clarity

The Clarity Node-S is a low-cost optical particle counter that measures PM₂.₅ via laser light scattering, with raw measurements recorded at minute-level temporal resolution [45]. For this study, hourly aggregated PM₂.₅ data were downloaded from the OpenAQ measurements API [46] for the period 2022–2025. These data were collocated with AirNow regulatory-grade monitors and used as the primary LCS input to develop and train calibration models.

3.2.3. Meteorological Variables

Hourly meteorological variables, including 2m temperature, 2m relative humidity, 10m wind speed, and pressure, were obtained from the National Oceanic and Atmospheric Administration (NOAA) High-Resolution Rapid Refresh (HRRR) model, a real-time 3-km resolution hourly updated atmospheric model covering the contiguous United States. HRRR data were downloaded from the NOAA archive hosted on Amazon Web Services (AWS) Simple Storage Service (S3) (s3://noaa-hrrr-bdp-pds) using the 2D surface-level fields at a 3-km spatial resolution. Relative humidity and temperature are the primary meteorological drivers of the bias in the LCS, as high humidity causes hygroscopic particle growth that inflates optical scattering signals, while temperature affects sensor electronics and particle composition, both of which systematically distort PM₂.₅ readings [47,48]. These variables were extracted at the collocated sensor locations and temporally matched to hourly PM₂.₅ observations.

3.2.4. Ancillary Variables

Land use and land cover (LULC) data were obtained from the National Land Cover Database (NLCD), produced by the United States Geological Survey (USGS) at 30m spatial resolution [49]. Federal Information Processing Standards (FIPS) codes were used to identify county-level administrative boundaries for each sensor location [50]. Geographic coordinates, including latitude and longitude, were included to enable spatial matching between collocated sensor pairs. Temporal categorical variables such as hour of day, day, month, season, day of year (DoY), day of week (DoW), and week of year (WoY) were derived from each observation's timestamp to capture diurnal, weekly, and seasonal cycles in PM₂.₅ concentrations.

4. Methodology

Figure 3 illustrates the overall research workflow, beginning with data acquisition from OpenAQ LCS and EPA regulatory monitors, followed by data collocation, preprocessing, and train-test splitting. The processed data are used to train a Transformer-based calibration model, which produces calibrated PM₂.₅ estimates. These calibrated PM₂.₅ estimates are then used to quantify uncertainty and assess reliability under wildfire events.

4.1. Data Collocation

Raw PM₂.₅ measurements from Clarity sensors were first subjected to outlier removal using a rolling standard deviation filter. Records with a rolling standard deviation of zero were excluded to remove inactive or non-reporting sensor readings. Following outlier removal, Clarity sensors were spatiotemporally collocated with the nearest AirNow regulatory-grade monitors at distance thresholds of 1 km, 5 km, and 10 km, retaining only sensor pairs with a Pearson correlation coefficient of 0.4 or higher to ensure a minimum level of measurement agreement before calibration.

Table 1 summarizes the collocated sensor pair statistics across both study areas. In the Northeast, the number of collocated pairs increased from 4 at 1 km to 19 at 10 km, with mean correlations ranging from 0.87 at 1 km to 0.81 at 10 km. In California, pairs increased from 11 at 1 km to 27 at 10 km, with mean correlations ranging from 0.80 at 1 km to 0.77 at 10 km. Across both study areas, the correlation declined as sensor separation distance increased. In contrast, the number of available pairs increased, reflecting a trade-off between spatial proximity and the availability of collocated sensor pairs for model training.

4.2. Data Preprocessing

This study applied several preprocessing steps to the collocated dataset before training, including gap-filling for missing values, data splitting, feature scaling, and sequence construction. Missing PM₂.₅ values were imputed using an XGBoost-based model trained separately for each region, achieving R² of 0.88 and RMSE of 2.81 µg/m³ for California, and R² of 0.89 and RMSE of 1.43 µg/m³ for the Northeast [51]. FIPS codes and LULC were incorporated as categorical variables to capture county-level administrative context and surface environment characteristics at each sensor location. The collocated dataset was then partitioned using a time-based split with 70% of the data used for training, 10% for validation, and 20% for testing, ensuring that all sets are temporally non-overlapping, preventing data leakage, and reflecting real-world deployment conditions. Input features were subsequently normalized using a standard scaler to ensure consistent scaling across continuous variables. Finally, the data were organized into sequential input windows of 23 preceding hourly observations, with each sequence used to calibrate the current-hour PM₂.₅ measurement. The sequence length of 23 was selected to capture near-diurnal temporal dependencies while maintaining computational efficiency [52].

4.3. Transformer Model Architecture

The proposed calibration model is based on a transformer architecture, which was introduced by [53]. Transformers capture long-range temporal dependencies through self-attention mechanisms while processing entire input sequences in parallel. This makes them well-suited for PM₂.₅ calibration, where capturing temporal patterns across the input window is essential [54]. The proposed model takes continuous and categorical spatio-temporal variables as input, producing a multi-step sequence of calibrated PM₂.₅ estimates. A schematic overview of the model architecture is presented in Figure 4.

4.3.1. Inputs/Embedding

Categorical temporal variables, including hour, day, season, month, DoY, DoW, and WoY, as well as location indicators, are represented using learnable embedding layers that map each category to a continuous vector representation. These embeddings allow the model to capture relationships among discrete temporal and spatial features. The resulting categorical embeddings are concatenated with continuous input features, including temperature, pressure, and humidity, to form the complete input sequence. This combined representation allows the model to learn from both discrete and continuous spatiotemporal information jointly.

4.3.2. Positional Encoding

Learnable positional embeddings are added to the projected sequence representation to preserve temporal relationships within the input data. Since the Transformer architecture does not inherently encode sequence order, positional information must be explicitly incorporated into the input representations [53]. These embeddings provide information about the position of each element in the sequence, enabling the model to distinguish between different temporal arrangements. This allows the self-attention mechanism to account for sequence order, enabling the model to capture temporal structure during both training and inference.

4.3.3. Encoder

The encoder consists of a stack of identical encoder layers. Each encoder layer includes multi-head self-attention followed by a feed-forward network. Residual connections and layer normalization are applied around both sublayers [53].

In the self-attention sub-layer, the input representation is linearly projected into query (Q), key (K), and value (V) matrices. The relationships between different positions in the sequence are computed using scaled dot-product attention, defined as Equation (1)

$$\mathrm{Attention}\left(Q,K,V\right)=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(1)

where Q, K, and V denote the query, key, and value matrices, and dk represents the dimensionality of the key vectors. The resulting attention outputs from multiple heads are concatenated and passed to the feed-forward network within the encoder layer. The encoder produces a transformed sequence representation that serves as memory for the decoder.

4.3.4. Decoder

The decoder also consists of stacked decoder layers. Each decoder layer includes self-attention, encoder-decoder attention, and a feed-forward network, with residual connections and layer normalization applied after each component. The decoder processes the encoded sequence representation to generate the final calibrated PM₂.₅ estimates.

4.4. Model Training

The decoder output is passed through a hidden linear projection layer to produce the predicted mean (μ) and variance (σ²) of PM₂.₅ concentrations. Model parameters are optimized using a hybrid loss function that combines Gaussian negative log-likelihood (NLL) and a weighted Huber loss [55]. The NLL term enables the model to capture aleatoric uncertainty by penalizing errors relative to the predicted variance. At the same time, the weighted Huber loss improves robustness to extreme PM₂.₅ values by reducing the disproportionate influence of large errors. The combined loss function, defined in Equation (2), balances uncertainty calibration with robustness to high PM₂.₅ outliers

$$L=\alpha \cdot L_{\mathrm{NLL}}+\left(1-\alpha \right)\cdot L_{\mathrm{Huber}}$$

(2)

The Gaussian Negative Log-Likelihood loss is defined as Equation (3):

$$L_{\mathrm{NLL}}={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N\left[{\displaystyle \frac{1}{2}}\log {\mathrm{\sigma }}_i^2+{\displaystyle \frac{{\left(y_i-{\mathrm{\mu }}_i\right)}^2}{2{\mathrm{\sigma }}_i^2}}\right]$$

(3)

where ${\mu }_i$and ${\sigma }_i^2$are the predicted mean and variance for the sample $i$, and $\alpha =0.5$ balances the two loss components.

4.5. Hyperparameters Optimization

Hyperparameter optimization was performed using random search over a predefined configuration space [56]. The search space included embedding dimensions, the number of attention heads, the number of layers, the feed-forward dimension, the dropout rate, the learning rate, and the batch size. Each configuration was trained with early stopping based on validation R², and the best-performing configuration was selected for final evaluation. The number of attention heads (8), number of layers (5), feed-forward dimension (1024), and optimizer (AdamW) remained consistent across all configurations, while the embedding dimension, batch size, dropout rate, and number of epochs varied across regions and distance thresholds, with the final configurations reported in Table 2.

4.6. Uncertainty Quantification

In this study, two complementary UQ methods are considered: MCD, a model-based method grounded in approximate Bayesian inference, and GCP, a distribution-free framework that provides statistically valid prediction intervals.

MCD, introduced by Gal and Ghahramani [57] provides an efficient framework for estimating predictive uncertainty in deep neural networks by interpreting dropout as a Bayesian approximation to a probabilistic deep Gaussian process. Under this framework, multiple stochastic forward passes are performed during inference with dropout active, producing a distribution of predictions from which the predictive mean and variance are derived. In this study, MCD is extended to jointly capture both epistemic and aleatoric uncertainties by predicting the mean and variance of the target distribution, with the training objective modified to a hybrid NLL loss to enable learning of input-dependent uncertainty. Equation (4) defines the total predictive uncertainty as the sum of epistemic uncertainty and aleatoric uncertainty.

$${\mathrm{\sigma }}_{\mathrm{total}}^2=\underset{{\mathrm{\sigma }}_{\mathrm{epistemic}}^2}{\underbrace{\mathrm{Va}{\mathrm{r}}_{\mathrm{t}}\left({\mathrm{\mu }}_t\right)}}+\underset{{\mathrm{\sigma }}_{\mathrm{aleatoric}}^2}{\underbrace{{\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{\mathrm{\sigma }}_t^2}}$$

(4)

where ${\mu }_t$ and ${\sigma }_t^2$ denote the predicted mean and variance obtained from the $t$-th stochastic forward pass with dropout enabled, ${\mathrm{V}\mathrm{a}\mathrm{r}}_t\left({\mu }_t\right)$ quantifies epistemic uncertainty across all stochastic predictions, and $T$ is the total number of stochastic forward passes, set to 30 in this study.

GCP is a model-agnostic UQ framework that extends conformal prediction to spatial prediction tasks [14]. Conformal prediction constructs prediction intervals from a calibration dataset by computing nonconformity scores, typically defined as residuals, and generates intervals with valid marginal coverage under minimal distributional assumptions [16]. The GCP incorporates geographic weighting to account for spatial heterogeneity and covariate shift through a Gaussian distance-decay kernel. It assigns greater weight to nearby calibration points and lower weight to distant ones, where the weight assigned to each calibration point is defined in Equation (5):

$$w\left(X_i\right)=K\left({\displaystyle \frac{\left|l_i-l_{n+1}\right|}{b}}\right)$$

(5)

where $l_i$is the location of the calibration point $i$, $l_{n+1}$is the test location, and $b$ is the bandwidth parameter controlling the rate of distance decay. The neighborhood included all calibration points without a fixed spatial threshold, and the bandwidth parameter was held constant across all test locations, yielding locally adaptive prediction intervals that reflect spatial heterogeneity.

4.7. Evaluation Metrics

To comprehensively evaluate both calibration performance and uncertainty reliability, this study employs a combination of standard regression metrics and uncertainty-specific metrics. While traditional metrics assess how well the calibration model matches observed PM₂.₅ concentrations, UQ metrics evaluate the reliability of the uncertainty intervals.

4.7.1. Calibration Accuracy Metrics

Standard regression metrics, including R², MAE, MSE, and RMSE, were used to evaluate calibration accuracy, as defined in [39].

4.7.2. Uncertainty Quantification Metrics

Expected Calibration Error (ECE) was used to quantify the difference between predicted confidence and observed coverage, as defined in [39]. Additionally, Prediction Interval Coverage Probability (PICP) and Prediction Interval Width (PIW) were used to evaluate the uncertainty intervals, as defined in Equations (6) and (7), respectively.

$$\mathrm{PIW}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n\left(u_i-l_i\right)$$

(6)

Where l_i and u_i denote the lower and upper bounds of the prediction interval for observation i.

$$\mathrm{PICP}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^{n}1\left(l_i<y_i<u_i\right)$$

(7)

where the indicator function equals 1 if the observed value lies within the interval and 0 otherwise.

5. Results

5.1. Statistical Analysis

Exploratory statistical analysis was conducted on PM₂.₅ concentrations from both AirNow and Clarity sensors across two study areas. Boxplots of PM₂.₅ concentrations were used to summarize the central tendency and variability of the data from both sensor types across three distance thresholds (1 km, 5 km, and 10 km), as shown in Figure 5 and Figure 6. In California (Figure 5) the two datasets show similar central tendencies, with median values of 5.1 µg/m³ for AirNow and 4.77 µg/m³ for Clarity. The interquartile ranges are comparable, spanning 3.1-8.2 µg/m³ for AirNow and 2.89-8.07 µg/m³ for Clarity. Despite these similarities, AirNow shows greater variability, with a standard deviation of 6.05 µg/m³ compared to 4.78 µg/m³ for Clarity. AirNow reaches a maximum of 401.1 µg/m³, while Clarity values reach a maximum of 320 µg/m³. The timing of these extreme observations aligns with the 2025 Palisades and Eaton Fires, indicating that these outliers correspond to real pollution events rather than measurement noise.

In the Northeast United States (Figure 6) both sensor types again show comparable central tendencies, though at lower concentrations than in California. AirNow reaches a maximum of 150 µg/m³, while Clarity reaches a maximum of 102 µg/m³, with extreme observations corresponding to smoke transport from Canadian wildfires [58].

To further examine the distributional characteristics, histograms of PM₂.₅ concentrations were analyzed for both AirNow and Clarity sensors across the two study areas, as shown in Figure 7 and Figure 8. In California (Figure 7), both AirNow and Clarity exhibit a strongly right-skewed distribution, with the majority of observations concentrated between approximately 3 and 8 µg/m³. Both sensors report similar typical PM₂.₅ values, with long upper tails corresponding to infrequent but extreme pollution events associated with the 2025 Palisades and Eaton Fires. In the Northeast United States (Figure 8), a similar right-skewed distribution is observed. However, extreme values are notably lower than in California, with upper-tail observations corresponding to smoke transport from Canadian wildfires.

5.2. Model Performance

5.2.1. California

Table 3 presents the model performance metrics across three spatial distance thresholds for California. The model demonstrates strong calibration performance at shorter distances, with test R² values of 0.89, 0.88, and 0.84 at the 1 km, 5 km, and 10 km thresholds, respectively, indicating a gradual decline in accuracy with increasing spatial separation. This decline suggests that increasing spatial separation introduces greater variability and reduces the model's ability to capture localized PM₂.₅ patterns. The gap between training and test performance remains relatively small at shorter distances, indicating limited overfitting when sensors are closely collocated. At larger distances, the increase in error metrics and bias (to 0.2606 µg/m³ at 10 km) suggests reduced generalization and greater difficulty in modeling spatial heterogeneity. These results confirm that calibration accuracy is sensitive to spatial distance, with closer sensor pairs yielding more reliable and consistent predictions.

Figure 9 compares uncalibrated and calibrated PM₂.₅ predictions against reference AirNow observations. The uncalibrated model (Figure 9a) shows substantial dispersion relative to the 1:1 line, with an R² of 0.601, an RMSE of 5.42 µg/m³, and a positive bias of 0.55 µg/m³, indicating systematic overestimation. In contrast, the calibrated model (Figure 9b) exhibits a much tighter alignment with the 1:1 line, with R² improving to 0.893 and RMSE reducing to 2.81 µg/m³. Notably, calibration also substantially reduces the bias from 0.55 to 0.12 µg/m³, demonstrating the model's effectiveness in correcting systematic overestimation of PM₂.₅ concentrations.

5.2.2. Northeast

Table 4 presents the model performance metrics for the Northeast United States across three spatial distance thresholds. Model performance remains strong across all distance thresholds, with only modest increases in error as spatial separation increases. At the 1 km threshold, the model achieves a test R² of 0.91 and RMSE of 1.38 µg/m³, indicating accurate calibration under closely collocated sensor conditions. Compared to California, error metrics in the Northeast are substantially lower across all thresholds, suggesting more consistent and homogeneous PM₂.₅ conditions in this region [59]. As spatial distance increases, performance declines modestly, with test R² decreasing from 0.91 at 1 km to 0.88 at 10 km and RMSE increasing from 1.38 to 1.63 µg/m³. The gap between training and test performance remains small across all distance thresholds, suggesting minimal overfitting and strong generalization. Bias values are consistently negative across all thresholds (ranging from -0.1245 to -0.3200 µg/m³), indicating a systematic tendency for the model to slightly underestimate PM₂.₅ concentrations, though the magnitude remains small.

Figure 10 compares uncalibrated and calibrated predictions against reference observations. The uncalibrated model (Figure 10a) shows substantial dispersion from the 1:1 line, with relatively low explanatory power (R² = 0.402) and higher error (RMSE = 3.60 µg/m³). In contrast, the calibrated model (Figure 10b) demonstrates a significant improvement, with R² increasing to 0.913 and RMSE decreasing to 1.37 µg/m³. Additionally, the reduction in bias from -0.98 µg/m³ to -0.12 µg/m³ highlights the calibration model's effectiveness in correcting systematic underestimation. Overall, these results demonstrate that calibration substantially improves both accuracy and reliability in the Northeast region, with stronger performance compared to California, particularly at larger spatial distances. This is likely because the Northeast has lower PM₂.₅ concentrations with less spatial variability, which reduces prediction difficulty [59], whereas California's complex terrain and episodic wildfire smoke introduce greater concentration heterogeneity that challenges model generalization [60].

5.3. Comparison of MCD and GCP

To further quantify the reliability and practical usefulness of the uncertainty estimates, calibration metrics including PICP, PIW, and ECE were evaluated across multiple confidence levels.

5.3.1. California

Table 5 presents calibration and uncertainty-interval metrics, such as PICP, PIW, and ECE, for MCD and GCP approaches across confidence levels at the 1 km distance threshold in California. The MCD approach demonstrates strong calibration performance, with observed PICP values closely matching the nominal confidence levels. At lower confidence levels, the model exhibits slight underconfidence, with observed PICP values exceeding the nominal levels (e.g., at the 50% confidence level, the observed PICP is 0.55 compared to the ideal of 0.50). At higher confidence levels, however, the trend reverses, with mild undercoverage observed (e.g., at the 99% confidence level, the observed PICP is 0.97 compared to the ideal of 0.99). The relatively low ECE value of 0.02 further confirms that the model is well-calibrated. The prediction intervals widen as expected with increasing confidence levels, with PIW values ranging from 2.37 µg/m³ at the 50% level to 9.12 µg/m³ at the 99% level, reflecting the model's ability to scale uncertainty with confidence appropriately.

The GCP uncertainty approach was evaluated using the same uncertainty metrics to assess its calibration performance. GCP produces values that are generally close to the nominal confidence levels but exhibits consistent undercoverage across all levels. For example, at the 80% and 90% confidence levels, observed PICP values of 0.76 and 0.86 indicate mild undercoverage. The ECE of 0.03 suggests good overall calibration, though slightly worse than MCD. In terms of prediction interval width, GCP produces narrower intervals at lower confidence levels (e.g., 1.67 µg/m³ at 50% vs 2.37 µg/m³ for MCD) but wider intervals at the 99% confidence level (12.90 µg/m³ vs 9.12 µg/m³), suggesting that GCP expands its intervals more aggressively at higher confidence levels to compensate for undercoverage.

The calibration curves for MCD and GCP at the 1 km distance threshold in California are presented in Figure 11. The MCD calibration curve lies predominantly above the ideal diagonal, indicating slight overcoverage at lower confidence levels where the observed PICP values exceed nominal targets. In contrast, the GCP calibration curve consistently falls below the diagonal across the full confidence range, reflecting systematic undercoverage. Notably, while MCD achieves a lower ECE (0.02 vs 0.03), its miscalibration area (0.03) is higher than that of GCP (0.02), suggesting that GCP's curve adheres more closely to the ideal diagonal across the full confidence range despite its slightly higher point-level deviation.

5.3.2. Northeast

Table 6 presents the PICP, PIW, and ECE metrics for MCD and GCP approaches at the 1 km distance threshold in the Northeast. The MCD approach demonstrates good calibration performance, with observed PICP values generally aligning with nominal confidence levels. However, at the 50% confidence level, notable overcoverage is observed (observed PICP of 0.67 vs. ideal of 0.50), while higher confidence levels remain well calibrated (e.g., 90%: 0.91, 99%: 0.97). The ECE of 0.06 is higher than the value observed in California (0.02), indicating a slightly weaker overall calibration in the Northeast. PIW values are consistently lower than in California, ranging from 1.23 µg/m³ at 50% to 4.74 µg/m³ at 99%, suggesting tighter prediction intervals in this region.

The GCP approach in the Northeast shows consistent undercoverage across all confidence levels, with observed PICP values falling short of the nominal levels. Undercoverage is most pronounced at lower confidence levels, with observed PICP values of 0.47, 0.71, and 0.82 at the 50%, 80%, and 90% confidence levels, respectively. The ECE of 0.06 matches that of MCD and is higher than that observed in California, indicating weaker uncertainty calibration in the Northeast. GCP produces notably narrower prediction intervals compared to MCD across all confidence levels (e.g., 0.66 µg/m³ vs. 1.23 µg/m³ at 50%), except for the 99% confidence level (6.01 µg/m³ vs. 4.74 µg/m³), suggesting that GCP expands its intervals more aggressively at higher confidence levels to mitigate its systematic undercoverage.

The calibration curves for MCD and GCP at the 1 km distance threshold in the Northeast are presented in Figure 12. The MCD calibration curve (Figure 12a) lies above the ideal diagonal across much of the confidence range, indicating overcoverage and a higher miscalibration area of 0.08. In contrast, the GCP calibration curve (Figure 12b) falls below the diagonal, indicating systematic undercoverage, but exhibits a lower miscalibration area of 0.03 and remains closer to the ideal diagonal overall. The largest deviations for GCP occur at mid-to-high confidence levels, particularly around the 80% and 90% confidence bounds.

5.4. UQ During Wildfire vs Normal Days

5.4.1. Wildfire Days

Figure 13a–c presents the calibration results for a California wildfire event at 29.40 m sensor separation, characterized by a PM₂.₅ spike exceeding 350 µg/m³. The MCD uncertainty band expands substantially during the peak, achieving 71% coverage of the ground-truth observations during the event (Table 7). In contrast, the GCP interval remains comparatively narrow during the event, resulting in lower coverage.

Figure 13d–f presents the corresponding results for a Northeast wildfire event at 6,798.84 m sensor separation, with PM₂.₅ concentrations reaching approximately 60 µg/m³. In this case, the MCD uncertainty band remains relatively narrow, achieving only 44% coverage of ground-truth observations during the event (Table 7). Conversely, GCP expands its prediction intervals during the event, achieving 83% coverage (Table 7). These results demonstrate that the relative performance of MCD and GCP varies with the spatial and environmental characteristics of wildfire events.

5.4.2. Normal Days

Figure 14 presents the time-series calibration of Clarity PM₂.₅ against AirNow ground truth under normal (non-wildfire) conditions in California (a–c) at 2.92 m and the Northeast (d–f) at 83.69 m spatial separation. Under normal conditions, both methods demonstrate strong calibration performance, with calibrated predictions closely tracking the ground truth across both regions. In California (Figure 14b), the MCD uncertainty band is narrow and efficient, capturing 81% of ground truth observations, while the GCP interval in Figure 14c is consistently wider throughout the time series, capturing 96% of observations.

In the Northeast (Figure 14e), MCD similarly produces narrow, efficient intervals with 73% coverage, while GCP in Figure 14f maintains wider intervals throughout, achieving 92% coverage. The wider GCP intervals reflect its residual-based calibration framework, which prioritizes coverage reliability over interval sharpness [61]. In contrast, MCD estimates uncertainty through stochastic forward passes, producing intervals that more closely follow local prediction variability. Overall, both methods track the ground truth well under normal conditions, with GCP providing higher coverage at the cost of wider intervals and MCD offering more efficient but less conservative uncertainty estimates.

5.5. Spatial Uncertainty

Figure 15 illustrates the spatial distribution of GCP-derived geospatial uncertainty across California at the 1 km distance threshold. The uncertainty varies considerably across sensor locations, ranging from approximately 1 to 6 µg/m³, reflecting the spatially adaptive nature of the GCP framework. Sensors located in the San Francisco Bay Area and along the central coastal region exhibit lower uncertainty values (1–2 µg/m³), suggesting that the model performs more consistently in these areas, likely due to higher sensor density and more homogeneous PM₂.₅ conditions. In contrast, the sensor in the southern coastal region near San Diego exhibits the highest uncertainty (approximately 6 µg/m³), indicating greater prediction variability, which may be attributed to sparse calibration data in the surrounding area or distinct local PM₂.₅ dynamics [62].

6. Discussion

6.1. Comparison of Uncertainty Performance

GCP generally exhibited calibration curves that remained closer to the ideal diagonal than MCD, indicating more consistent calibration across confidence levels [14]. The contrasting overcoverage of MCD and undercoverage of GCP suggest that the two methods characterize predictive uncertainty differently [14,57]. Regional differences indicate that uncertainty calibration is influenced by local spatial characteristics, with both methods exhibiting weaker calibration performance in the Northeast than in California [63]. The Northeast results show that similar ECE values do not necessarily correspond to similar calibration performance. Miscalibration area and calibration curves provided additional insight into uncertainty reliability beyond that captured by ECE alone. These findings emphasize the importance of evaluating uncertainty calibration using multiple complementary metrics rather than relying on a single performance measure.

The event-based analysis revealed important differences in the behavior of MCD and GCP under wildfire and normal conditions. MCD and GCP exhibited a trade-off between interval efficiency and coverage reliability across the evaluated scenarios. Neither method consistently outperformed the other across all wildfire events, with MCD achieving higher coverage during the California wildfire event and GCP achieving substantially higher coverage during the Northeast wildfire event. These differences suggest that the relative performance of UQ methods depends strongly on the spatial and environmental characteristics of the event. The contrasting behavior of the two methods may reflect their different interval construction mechanisms, with MCD deriving uncertainty from model-estimated variance and GCP calibrating interval width using local residual information [14,57]. Under normal conditions, GCP consistently achieved higher coverage through wider prediction intervals, whereas MCD produced narrower and more efficient intervals. Collectively, these findings indicate that method selection should balance interval efficiency and coverage reliability according to the expected degree of spatial separation and distribution shift.

6.2. Comparison with Existing Studies

Table 8 compares this study with recent LCS calibration studies in terms of study area, sensor type, model, performance metrics, wildfire evaluation, and UQ. Compared to existing studies, this study achieved competitive baseline performance (R2=0.89 in California and 0.91 in the NE), comparable to high-performing urban calibration models [12,64] and outperforming tree-based spatial approaches [38,65]. Previous calibration studies, such as Park, Yoo, Park and Lee [12] primarily evaluated calibration performance under standard environmental conditions and did not explicitly assess model reliability during wildfire-driven distribution shifts. Similarly, Delp and Singer [29] addressed wildfire smoke impacts using fixed empirical scaling relationships rather than adaptive deep learning calibration. Furthermore, this study addresses an important gap in the PM₂.₅ calibration literature related to predictive reliability. Previous calibration studies either lacked formal UQ entirely or limited uncertainty analysis to descriptive post-hoc variability assessments [65] or distance-dependent error analyses [38]. In contrast, this study introduces a comparative evaluation of model-based and model-agnostic uncertainty quantification frameworks for low-cost PM₂.₅ sensor calibration. Another major contribution of this work is the systematic evaluation of uncertainty reliability under both normal and extreme pollution conditions, extending calibration assessment beyond predictive accuracy alone. Practically, these prediction intervals can help air quality agencies determine when LCS PM₂.₅ estimates are reliable or uncertain, improving confidence in exposure assessment, wildfire smoke monitoring, and public health decision-making during extreme pollution events.

6.3 Future Directions

Several extensions of this work could further improve calibration robustness and broaden its applicability. Future work should examine the operational deployment of calibrated LCS, where the trustworthiness and reliability of their uncertainty estimates are critical for air-quality decision-making [66]. Such deployment would help translate calibrated estimates into actionable exposure information, extending dependable PM₂.₅ monitoring to underserved communities and supporting timely health advisories during extreme pollution episodes [67]. In this context, method selection should balance interval efficiency against coverage reliability, prioritizing coverage where missed exposures carry the greatest risk [68]. Second, data augmentation strategies, such as generating synthetic sensor observations for under-represented pollution regimes, could improve model robustness and reduce predictive uncertainty during extreme events like wildfires [69]. Targeted augmentation of high-PM₂.₅ episodes is particularly promising given the upper-tail underestimation observed in the wildfire-period results. Third, calibrated LCS could function as a high-resolution observational layer within digital twin systems, densifying the spatial coverage of sparse regulatory networks [70]. Through near-real-time data assimilation, these observations could continuously update the digital twin state, enabling scenario-based forecasting and adaptive air-quality decision-making [71]. Together, these directions point toward a next generation of uncertainty-aware, spatiotemporally adaptive calibration frameworks that can deliver trustworthy PM₂.₅ estimates across diverse environmental conditions.

7. Conclusion

This study presented a systematic evaluation of UQ methods for PM₂.₅ calibration using LCS data, with a focus on reliability under spatial variability and wildfire-induced distribution shift. A transformer-based calibration model was combined with two complementary UQ methods, MCD and GCP, to assess predictive reliability across California and the Northeast United States. The calibration model achieved strong performance at short spatial distances, with test R² values of 0.89 and 0.91 at the 1 km threshold in California and the Northeast, respectively, and accuracy generally decreasing as sensor separation increases, highlighting the sensitivity of calibration to spatial heterogeneity. Both approaches produced reliable uncertainty estimates but differed in how uncertainty is represented based on spatial and temporal constraints. In terms of reliability, GCP generally exhibited calibration curves that more closely followed the ideal diagonal than MCD across the study regions. In practice, MCD provides narrow, efficient intervals under normal conditions and scales effectively during highly localized pollution anomalies. At larger spatial separations, GCP generally achieved higher coverage through wider prediction intervals. These findings demonstrate that uncertainty behavior is highly context-dependent and that event-specific analyses reveal patterns that may not be apparent from aggregate performance metrics alone. To conclude, this study emphasizes the importance of incorporating UQ into air quality workflows to ensure model trustworthiness and reliability under extreme conditions where accurate predictive confidence is essential. Future work should explore data augmentation strategies for high PM₂.₅ events, adaptive calibration approaches for GCP, and hybrid frameworks that combine the complementary strengths of model-based and model-agnostic uncertainty methods.