Portable Sensors for Dynamic Exposure Assessments in Urban Environments: State of the Science

1. Introduction

Air quality has improved significantly over the past decades. Yet exposure to particulate matter and nitrogen dioxide in Europe still cause an estimated 253,000 and 52,000 premature deaths per year [1]. Moreover, continuous worldwide urbanization results in megacities with intrinsic hotspots, highlighting the importance of proper air pollution monitoring. Currently, the exposure of the population to air pollution is still determined based on the home address (=static exposure). However, research has shown that people are exposed to the highest air pollution peaks at times when they are mobile (e.g. during commutes). Studies using activity-based models or personal monitors demonstrated that the transport activity, although short in duration, can be responsible for quite a large part of integrated personal exposure to combustion-related pollutants [2,3,4,5,6]. Therefore, their exposure during transport must also be taken into account when calculating personal exposure to air pollution, i.e. dynamic exposure. Research based on an extensive dataset of 20,000 citizens previously showed that this dynamic exposure estimation is often (64% of the individuals) higher than the respective static residence-based exposure [7]. Therefore, a better approximation of citizens dynamic exposure to air pollution is needed through modelling and/or monitoring tools. To assess exposure in transport environments on a wider scale, mobile monitoring systems are needed that (i) can easily be used by study participants (e.g. citizens) and (ii) produce reliable data.

Recent advances in sensor and Internet of Things (IoT) technologies have resulted in a wide range of commercially available “lower-cost” sensor systems that allow for quantification of urban pollutants, e.g., particulate matter (PMx), nitrogen dioxide (NO₂) and ozone (O₃), at unprecedented scale [8]. Portable air quality sensors enable quantification of dynamic exposure, while raising awareness among citizens about their personal exposure driving behaviour change [9,10,11,12,13]. Moreover, the obtained mobile data can be used to construct urban exposure maps offering policy makers the right tools for evidence-based policy measures [8,14,15,16,17,18,19]. As Helbig et al. [20] stated; wearable sensing has two aspects: firstly, the exposure of an individual is recorded, and secondly, individuals act as explorers of the urban area. While many stationary sensor systems have been evaluated and benchmarked in previous years [21,22], mobile sensor systems have different requirements, e.g. power autonomy (battery), a high monitoring resolution and accurate positioning (GPS). Also, the sensor signal noise and between-sensor variability should be low enough to be able to measure the spatial concentration variability at a high temporal resolution (with multiple sensors). Today, many commercially available portable sensor systems are already on the market, but it is hard to determine they are fit-for-purpose. This is one of the first studies benchmarking commercially available portable sensor systems for mobile monitoring applications. The study includes the evaluation of the air quality sensor data quality performance in lab and field conditions, and during a mobile field test to evaluate GPS performance, impact of vibrations on the sensor signal and the overall potential to capture spatial variability in urban environments.

2. Materials and Methods

2.1. Sensor System Selection

Based on an earlier literature study and market survey on air quality sensors [23], a new literature study with focus on portable air quality sensors (~90 publications), sensor benchmarking results from independent research institutes (AIRlab (https://airlab.solutions/en/projets/challenge-microcapteurs-edition-2023-189), AQ-SPEC (https://www.aqmd.gov/aq-spec), SamenMeten (https://www.samenmeten.nl), EPA Air Sensor Toolbox (https://www.epa.gov/air-sensor-toolbox)) and sensor projects (BelAir [24], Snuffelfiets [25],…), a longlist of 39 sensor candidates was created. This longlist was narrowed down based on a set of predefined requirements (e.g. monitoring resolution, portability, localization, additional environmental variables like temperature and relative humidity), resulting in a shortlist of 12 portable sensor systems for which quotation requests were send out. Ultimately, 10 sensor systems were purchased (Table S1), of which 8/10 contained a PM_2.5 and PM₁₀ sensor, and 3/10 sensor systems contained an additional NO₂ sensor (SODAQ NO₂, DST Observair and 2BTech PAM). All 10 sensor systems can be regarded as portable air quality sensor systems, with autonomy (battery), data storage and/or transmission and localization (GPS).

Figure 1. Considered sensor systems (10) with on the upper picture (from left to right): PAM (2BTech), GeoAir, Observair (DST), SODAQ Air (SODAQ), PMscan (TERA), Open Seneca (Open Seneca) and ATMOTube Pro (ATMO). Lower pictures: SODAQ NO2 (left), Habitatmap Airbeam (middle) and BCmeter (right).

Figure 1. Considered sensor systems (10) with on the upper picture (from left to right): PAM (2BTech), GeoAir, Observair (DST), SODAQ Air (SODAQ), PMscan (TERA), Open Seneca (Open Seneca) and ATMOTube Pro (ATMO). Lower pictures: SODAQ NO2 (left), Habitatmap Airbeam (middle) and BCmeter (right).

It should be noted that BCmeter can be considered as a research prototype for stationary measurements (wifi, power cable), and was only evaluated in the field co-location campaign. In order to obtain a portable BCmeter, additional hardware/software development will be needed.

2.2. Benchmarking Protocol

The purchased sensor systems were evaluated under controlled (laboratory) and real-life (field) conditions (Figure 2). Field benchmarking included a mobile test on a cargo bike and a 3-month co-location campaign at a regulatory urban background (R801) air quality monitoring station in Antwerp, Belgium.

2.2.1. Laboratory Test Protocol

Laboratory tests were performed for both PM and NO₂.Test levels and test conditions for NO₂ were based on the CEN/TS 17660-1:2021. For PM we included a laboratory test to evaluate the potential of the sensor to measure the coarse fraction (PM_2.5–10 = PM₁₀- PM_2.5) because it is known that some low-cost sensor calculate PM₁₀ concentrations based on the measured concentrations of PM_2.5) and sensors can have various response characteristics regarding size selectivity [26,27]. For PM_2.5 and PM₁₀, we evaluated:

• Lack of fit (linearity) at setpoints 0, 30, 40, 60, 130, 200 and 350 µg/m³ (PM₁₀, dolomite dust). A Palas Particle dispenser (RBG 100) system connected to a fan-based dilution system and aluminium PM exposure chamber was used.
• Sensitivity of PM sensor to the coarse (2.5-10µm) particle fraction. We dosed sequentially 7.750 µm and 1.180 µm-sized monodisperse dust (silica nanospheres with density of 2 g/cm3) using an aerosolizer (from the Grimm 7.851 aerosol generator system connected to a fan-based dilution system and an aluminium PM exposure chamber with fans to have homogeneous PM concentrations.. This testing protocol is currently considered to be included in the CEN/TS 17660-2 (in preparation) on performance targets for PM sensors.

Based on the lack of fit results, the comparability against the reference is evaluated from the resulting linearity (R²), accuracy (%), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Bias Error (MBE) and Expanded Uncertainty (Uexp). As reference instrument, we used a Grimm 11-D with heated sampling inlet line (EDM 264, Grimm).

The comparability between the sensors can be regarded as the observed variability between sensors of the same type and is calculated by the between-sensor-uncertainty (BSU):

$${BSU}_{sensor}=\sqrt{\frac{\sum _{i=1}^n\sum _{j=1}^k{\left({sensor}_{ij}-{average}_i\right)}^2}{\left(n-1\right)}}$$

(1)

with n the number of sensors (3) and k the number of measurements.

In addition, we calculated the minimal and maximal observed Pearson correlation (r) and MAE (µg/m³) between the sensors of the same brand in order to evaluate the intra-sensor comparability.

For NO₂, we evaluated:

• Lack of fit (linearity) at setpoints of 0, 40, 100, 140 and 200 μg/m³.
• Sensor sensitivity to relative humidity at 15, 50, 70 and 90% (±5%) during stable temperature conditions of 20 ± 1°C.
• Sensor sensitivity to temperature at -5, 10, 20 and 30 °C (±3°C) during stable relative humidity conditions of 50 ± 5%
• Sensor cross-sensitivity to ozone (120 µg/m³) at zero and 100 µg/m³
• Sensor response time under rapidly changing NO₂ concentrations (from 0 to 200 µg/m³).

From the lack of fit tests, the comparability against the reference was evaluated from the resulting linearity (R²), accuracy (%), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Bias Error (MBE) and Expanded Uncertainty (Uexp).

In addition, we evaluated sensor stability (standard deviation at each setpoint) and intra-sensor comparability by calculating the between-sensor-uncertainty (BSU). As reference instrument, we applied a Thermo Scientific 42iQ-TL chemiluminescence monitor (Thermo Fisher, USA).

2.2.2. Mobile Field Test

The mobile field test aimed at testing the GPS accuracy of the sensor systems along a ~10 km trajectory within the varying urban landscape (street canyons, open parks, tunnels, …) of Antwerp, Belgium (Figure 3). GPS accuracy was evaluated by calculating the average horizontal distance (m) of the high-resolution mobile GPS measurements to a reference GPS track. The reference GPS track was determined by evaluating 3 different GPS platforms (TomTom Runner2, Garmin Edge 810 and Komoot smartphone application), and selecting the best performing one as reference GPS trajectory.

2.2.3. Field Co-Location Campaign

During the field co-location campaign, the considered sensor systems were exposed for a period of 3 months (7/9/2022-5/12/2022) to ambient pollutant concentrations in an actively-vented outdoor shelter, deployed on top (near the air inlets) of a regulatory urban background monitoring station (R801) in Antwerp, Belgium. The collected raw (RAW) and lab-calibrated (LAB CAL; linear calibration based on lack-of-fit) sensor data was subsequently evaluated for:

• Hourly data coverage (%)
• Timeseries plot: RAW & LAB CAL
• Scatter plot: RAW & LAB CAL
• Comparability between sensors: Between sensor uncertainty (BSU)
• Comparability with reference (hourly): R², RMSE, MAE, MBE
• Expanded uncertainty (non-parametric): Uexp (%)

In addition we evaluated the sensitivity of the sensors (R², RMSE, MAE, MBE) towards the (real-life) coarse particulate fraction (PM₁₀-PM_2.5) and the impact of a 2-week field co-location calibration (FIELD CAL; linear calibration for PM and multilinear for NO₂) on the resulting sensor performance and compared the field calibration performance to the lab calibration performance.

3. Results

3.1. Laboratory Test

3.1.1. PM

Due to the varying monitoring resolutions of the sensor systems (2 sec - 5min; Supplementary S1), all data was temporally aggregated to a 1-minute resolution and merged with the reference (Grimm 11D) data. The SODAQ Air and NO₂ apply a 5 minute resolution when stationary and change automatically to ~10 seconds when mobile, resulting in fewer datapoints in the laboratory test. The GeoAir experienced power supply issues during the lack-of-fit measurements (insufficient amperage from applied USB hubs), resulting in data loss for all sensors. Setpoint averages (µg/m³) were calculated from the most stable concentration periods (final 15 minutes of each 1-hour setpoint). From these setpoint averages, lack-of-fit curves were generated and regression coefficients (slope + intercept (y=a*x+b) and slope only (y=a*x)) and sensor accuracy (%) calculated. All results are shown per sensor system and subsequently presented in an overview table.

All sensor systems respond nicely to the increasing particle concentrations inside the PM exposure chamber, resulting in a generally good linearity between sensor and reference (R² = 0.96-1). Nevertheless, most of the sensor systems seemed to underestimate the actual PM_2.5 and PM₁₀ concentrations, while overestimating the PM₁ particle size fraction. Mean setpoint accuracy (mean of different setpoint accuracies) varied between 82-85% for PM₁, 63-69% for PM_2.5 and 28-31% for PM₁₀ (ATMOTUBE), 12-28% for PM₁, 76-84% for PM_2.5 and 45-51 for PM₁₀ (TERA PM SCAN), 80-86% for PM₁, 53-56% for PM_2.5 and 22-23 for PM₁₀ (OPEN SENECA), 31-94% for PM₁, 48-95% for PM_2.5 and 20-43 for PM₁₀ (SODAQ AIR), 60-77% for PM₁, 35-70% for PM_2.5 and 13-29 for PM₁₀ (SODAQ NO₂) and 63% for PM₁, 29% for PM_2.5 and 13% for PM₁₀ (2BTECH PAM). Calculated setpoint averages and derived lack-of-fit curves for PM₁, PM_2.5 and PM₁₀ can be found in Supplementary S2 and S3. Average regression coefficients (slope and intercept) and R² statistics are provided for each sensor system and particle size fraction.

From Figure 4, it can be observed that the between-sensor uncertainty (BSU) is larger for the SODAQ AIR (3.96 µg/m³) and NO₂ (no simultaneous data), when compared to ATMOTUBE (1.52 µg/m³), OPEN SENECA (1.21 µg/m³) and TERA PM (1.64 µg/m³). For the 2BTech PAM, this could not be evaluated as we had only one device available.

After applying a linear lab calibration (based on lack-of-fit regression coefficients), all sensor systems fell within expanded uncertainty <50% for PM_2.5, which is the data quality objective for indicative (Class 1) sensor systems (cfr. CEN/TS 17660-1 for gases).

Recent research showed that particle sensors exhibit low sensitivity in the coarse particle size range (2.5-10µm) [28,29]. Therefore, a test procedure was developed to evaluate sensor sensitivity to the coarse fraction and to evaluate if sensors really measure PM₁₀ rather than extrapolating it from the PM_2.5 signal. We expose the sensors to monodisperse dust (silica microspheres) of consecutively 7.75 µm and 1.18 µm (fine) diameters. We finetuned the settings of the aerosolizer to reach representative (~100-150 µg/m³) PM₁₀ concentrations by generating dust pulses every 30 seconds during a 5 minute period. The idea is to simulate conditions with mainly fine (‘Fine test cond.’) and mainly coarse aerosol (‘Coarse test cond.’) respectively. Two representative 5-minute periods (1 coarse test, 1 fine test) were subsequently selected and evaluated by calculating the dust composition (% coarse), PM₁₀, PM_2.5 and PM_coarse sensor/REF ratios and 2 relative change metrics (%):

• Relative change (%) in fractional (coarse vs fine) sensor/REF ratio during respective fine and coarse test conditions

$${Rel}_{{PM}_{fractional}}\left(\%\right)=\frac{\left(\frac{{PM}_{10-2.5 (sen, COARSE)}}{{PM}_{10-2.5 (REF, COARSE)}}-\frac{{PM}_{2.5 (sen, FINE)}}{{PM}_{2.5 (REF, FINE)}}\right)}{\frac{{PM}_{2.5 (sen, FINE)}}{{PM}_{2.5 (REF, FINE)}}}\ast 100$$

(2)

• Relative change (%) in PM₁₀ sensor/REF ratio between fine and coarse test conditions

$${Rel}_{{PM}_{10}}\left(\%\right)=\frac{\left(\frac{{PM}_{10 (sen,COARSE)}}{{PM}_{10 (REF, COARSE)}}-\frac{{PM}_{10 (sen, FINE)}}{{PM}_{10 (REF, FINE)}}\right)}{\frac{{PM}_{10 (sen, FINE)}}{{PM}_{10 (REF,FINE)}}}\ast 100$$

(3)

The sensor systems tend to visually pick up fine particle spikes but appeared far less responsive to the coarse fraction spikes (Figure 5). Note that in both fine and coarse generation spikes, PM_2.5 is present. Similar responses are observed between the different sensor systems, which is not surprising as all sensors are ultimately based on three original equipment manufacturer (OEM) sensors, namely Sensirion SPS30, Plantower PMS and TERA next-PM. From the calculated change ratios in Supplementary S4, the sensor/REF ratio changed significantly between the considered particle size conditions (73-100%), with all sensors showing very low sensitivity towards the coarse particle size fraction (PM_coarse sensor/REF ratio between 0-0.11 in Supplementary S4).

3.1.2. NO₂

For all sensors containing a NO₂ sensor (3/10), lack-of-fit tests were conducted on three days (12/8, 14/8 and 15/8) at concentrations ramping between 0 and 200 µg/m³ (Figure 6). Due to the varying monitoring resolutions of the sensor systems (2 sec - 5min), all data was temporally aggregated to 1-minute resolution and merged with the reference data (Thermo NO_x analyzer). Setpoint averages were calculated based on steady-state conditions (final 1.5-hour considering a 15-minute buffer period before each setpoint change). From these setpoint averages, lack-of-fit plots were generated, regression coefficients (slope + intercept (y=a*x+b) and slope only (y=a*x)), sensor stability (µg/m³) and setpoint accuracy (%) calculated. The SODAQ NO2 showed significant noise and data connectivity issues, resulting in a low stability (5-80 µg/m³) and setpoint accuracy (-113-254%). Moreover, sensor readings were inversely correlated (R²=0.03-0.18) to the actual NO₂ concentrations (Figure 6), with a poor between-sensor uncertainty (BSU) of 125 µg/m³. This out-of-the-box performance can be considered as inadequate. Potential calibration is hindered by the high signal noise, while sensor boxes showed connectivity issues and high BSU. The 2BTech PAM (only one unit available) was positively correlated with the generated NO₂ concentrations, with a mean setpoint accuracy of 72%, but exhibited significant noise and extreme peak values during the lack-of-fit test resulting in low sensor stability of 27 µg/m³. The DST Observair (one unit available) is not pre-calibrated by the supplier and relies on co-location calibration in the field. The uncalibrated sensor readings during the lack-of-fit test varied between -0.03 and 0.03 µg/m³ and showed a negative linear response to the increasing NO₂ concentration steps. Compared to the SODAQ NO2 and PAM, the Observair exhibits much lower signal noise, resulting in better stability (<0.01 µg/m³) and better calibration potential. After calibration, the expanded uncertainty (Uexp) of the Observair (65%) outperforms the observed accuracies of the SODAQ NO2 (415-490%) and PAM (80%). Nevertheless, the considered NO₂ sensors do not classify for the Class 1 uncertainty objective of <25% (CEN/TS 17660-1).

The impact from a changing relative humidity (0-50-75-90%) at zero and span concentration resulted in similar responses (Supplementary S7), with initial peak responses with every setpoint change followed by a subsequent stabilization (transient effect), under different levels of noisiness (Observair<PAM<SODAQ NO₂). Similar responses can be explained by the underlying OEM sensor (Alphasense NO2-B43F) which is similar for all NO₂ sensor systems. Similar transient effects (Supplementary S8) were observed under varying temperatures (-5, 10, 20 and 30°C), both at zero and span concentration.

To evaluate response time to rapidly changing NO₂ concentrations, sensors were placed in glass tubes that allowed for rapid concentration changes between 0-200 µg/m³ (Supplementary S9). The smaller volume of the glass tubes (compared to the NO₂ exposure chamber), only allowed evaluation of the Observair and PAM sensor as the SODAQ NO₂ boxes didn’t fit in the glass tubes. 30-minute setpoints (0 and 200 µg/m³) were considered and lab-calibrated sensor data was compared to the 1-minute data from the Thermo NOx analyzer. Averages and 90-percentiles (90% of max concentration) concentrations were determined for each 200 µg/m³ plateau, and the associated response time, i.e. time needed to reach 90% concentration was calculated for each sensor system (and reference analyzer). The resulting response times derived from the 3 consecutive 0-200 plateaus are provided in Supplementary S9 and varied between 1-2 minutes for the sensor systems and 3 minutes for the Thermo NOx reference analyzer.

An overview of the observed quantitative performance metrics for each of the sensor systems during the laboratory testing is provided in Table 1.

3.2. Mobile Field Test

All sensors were mounted on top (in the free airflow) of a cargo bike. Package sleeves were applied to damp vibrations of the cargo-bike whilst cycling. Besides the sensors, two mid-range instruments namely a Grimm 11D (PM; without heated inlet) and MA200 (BC) were placed inside the cargo bike with air inlets at the height of the sensors. Finally, the cargo bike was equipped with 3 different GPS instruments (Garmin 810 Edge, TomTom Runner 2, Komoot smartphone application). The TomTom track showed the highest monitoring resolution (1 sec) and horizontal accuracy and was, therefore, selected as reference track. The exhibited PM_2.5 concentration variability (measured by the Grimm) ranged between 4.8 and 133.3 µg/m³, while the BC (measured by the MA200) varied between 0.4 and 4.4 µg/m³ (Supplementary S10). While highest PM_2.5 concentrations were observed at a housing façade construction site, highest BC concentrations were obtained when cycling downwind of a busy highway (E313/E34). When plotting all sensor tracks on a map (Figure 7), the GPS accuracy performs visually better in open areas, compared to narrow and/or high street canyons. A higher height/width ratio seems to result in lower GPS accuracy. GPS accuracy deteriorates as well when moving through tunnels.

When calculating the average horizontal accuracy (m) as average distance to the reference track in QGIS (Figure 7), the horizontal accuracy was generally good achieving a <10m horizontal accuracy for all sensor systems (Supplementary S10). Highest horizontal accuracy (2.28 m) was obtained for the TERA PMscan, while the lowest horizontal accuracy (8.15 m) was observed for the GeoAir.

With regard to the measured raw sensor signals (PM/NO₂/BC), the mobile deployment (and related vibrations), did not seem to result in additional instrument noise or outliers when compared to stationary conditions. Moreover, similar hotspots were identified when comparing the sensor systems to the high-grade (MA200 and Grimm) monitors (Supplementary S12).

3.3. Field Co-Location Campaign

All sensor systems were deployed for 3 months (7/9/2022 – 5/12/2022) in an actively vented exposure shelter on top an urban background monitoring station (R801) in the city center of Antwerp (Figure 8). Sensor systems were evenly distributed across the three shelter levels. Different data storage and transmission protocols, including automatic cloud upload via GPRS/4G (SODAQ), internal SD-card storage (GeoAir), while some sensor systems relied on a smartphone application (TERA PMscan, ATMOTube) or a combination of these data transmission protocols (PAM, OpenSeneca, Airbeam, Observair). Some sensor systems were not designed for continuous, long-term monitoring. TERA PMscan relied on a smartphone application for operation, which resulted in forced automatic shutdowns by the smartphone software after some time (~1-2 days), and lack of continuous long-term data. The Observair relied on filter replacements for its BC measurement. As the filter saturated quickly, the instrument turns in error mode and did not collect any BC or NO₂ data. The BCmeter also relies on filter replacements. A dedicated 1.5 week campaign (16/11-30/11) was therefore set up to evaluate BC (and NO₂ from the Observair). The Airbeams arrived later and became operational on the 9^th of November. Sensor data was offloaded (remotely via web dashboards and on-site via SD card readout) weekly to avoid dataloss and a logbook was created to keep track of that status and encountered issues. From R801, we collected NO₂ (Thermo 42C; µg/m³), O₃ (Teledyne API400E; µg/m³), PM₁, PM_2.5, PM₁₀ (Palas FIDAS 200; µg/m³), BC (Thermo MAAP; µg/m³), relative humidity (%) and temperature (°C). The hourly data showed a good data availability with hourly (n=2132) data coverage of 96.7, 96.6 and 92.9% for, respectively, PM, BC and NO₂.

PM_2.5 concentrations ranged between 1-51 µg/m³ (mean =10.85 µg/m³), while NO₂ exhibited 2-111 µg/m³ (mean = 26 µg/m³). Atmospheric temperature varied between 1 and 27°C (mean = 13°C), while relative humidity was within 42 and 100% (mean=83.5%). Temporal pollutant variability reflects typical urban pollution dynamics (Supplementary S11), with morning and evening rush hour peaks for NO₂ and BC, slightly delayed PM peaks with a regional background character and O₃ that is photochemically formed at low NO₂ concentrations and high solar radiation conditions (inversely related to NO₂).

For each of the sensor systems, hourly data coverage, linearity (R²), accuracy, expanded uncertainty, impacts from lab and field calibration and sensor drift (sensor/REF ratio) over time was evaluated (Table 2). For PM sensor systems, the sensitivity towards the coarse particle fraction (PM_10-PM_2.5) and impact from respectively lab- and field calibrations were additionally evaluated. PM field calibration was based on slope/intercept derivation during a training period (first 2 weeks: 7/9/2022-21/9/2022) and evaluated based on the remaining 2.5 months of data (22/9/2022-5/12/2022). For NO₂ sensor systems, a multilinear field calibration model was trained with covariates for sensor response, temperature, RH and O_3, following earlier sensor calibration studies [30,31,32]. Model training was based on 2 weeks of co-location data and the calibration performance was tested on the remaining 2 months of test data. This multilinear field calibration outperformed the raw and lab-derived calibrations for all NO₂ sensor systems. Lab-derived calibrations did not hold in field conditions, which is not surprising as field conditions are different in terms of PM composition and meteorological conditions (temperature, relative humidity). Compared to the observed PM_2.5 performance in Table 2, performance decreases for PM₁₀ (R²=0.6-0.62, MAE=12.6 µg/m³) and the association is entirely lost (R²=0-0.01) when focusing on the coarse fraction (PM_coarse=PM₁₀-PM_2.5) confirming the lack of sensitivity in the coarse particle size fraction. For PM_2.5, general good correlations (R² = 0.7-0.9), varying accuracies (MAE=3-4.7 µg/m³) and low between-sensor uncertainties (0.1-0.7 µg/m³) were observed. The accuracy worsened by applying the lab calibration but was optimized further for all sensor systems based on the field calibration. No distinct aging effect (gradual deviation in sensor/REF ratio) was observed over the 3-month co-location period. An overview of the observed quantitative performance metrics based on the hourly-averaged data for each of the sensor systems during the field co-location campaign is provided in Table 2.

Hourly PM_2.5, NO₂ and BC timeseries of the considered sensor systems and reference data are provided in Figure 9.

4. Discussion

During this lab and field benchmarking campaign, we collected a lot of quantitative and qualitative evidence on the fit-for-purpose of current commercially-available dynamic exposure sensor systems. An overview is provided of the observed sensor system performance (hourly coverage, accuracy, R², MAE, BSU, stability, Uexp) for the considered pollutants under laboratory (Table 1) and real-world (Table 2) conditions.

For the considered PM sensor systems, out-of-the-box performance is already quite good and close to the Class 1 data quality objective (U_exp<50%). In addition, the sensors showed high precision, <0.4 µg/m³ in the lab and <0.6 µg/m³ in the field, which allows for multi-sensor (network) applications (e.g. [10,24,33]). Whether the obtained accuracy is sufficient to characterize PM gradients in urban environments (which are typically not that steep) will depend on city to city and should be further investigated. In our mobile field test, Grimm measurements showed PM_2.5 concentrations along the 10km trajectory ranging from 4.8 to 133 µg/m³. This exposure variability is, therefore, quantifiable by the considered sensor systems with MAE’s of 3-4.7 µg/m³. Highest accuracy was observed for PM₁, followed by PM_2.5 and PM₁₀. The considered sensor systems do not reliably detect the coarse particle size fraction. TERA is the only sensor system that seems to pick up some coarse particles (R²=0.3), while all other sensors show R² of ~0. Accuracy of PM sensors can be further improved by linear slope/intercept calibration. However, we showed that lab calibrations do not hold in the field, as previously shown in other studies [8,10,30,34]. A local field calibration (representative pollutant and meteorological environment) seems, therefore, crucial to obtain the most reliable sensor data. In general the assessed PM performance and observed sensitivities (drift/RH) are very similar between the benchmarked PM sensors, which can be explained by similar underlying sensor technology (Sensirion SPS30 + Plantower) and lack of applied factory-algorithms. Sensor systems showed elevated sensor/REF ratios with increasing relative humidity, as well documented in previous studies [30,35,36,37,38,39,40,41,42,43,44].

Regarding NO₂, out-of-the-box performance was unsatisfactory for direct application, as sensor systems suffered from noise (stability) and calibration (negative association) issues. Although 2BTech PAM showed the best raw performance, a higher but negative association (R²) and stability was observed for Observair. Following a linear laboratory calibration, best performance was, therefore, achieved for the Observair. Similar to the PM sensors, linear lab calibrations do not hold in the field. For NO₂, a local and multilinear field calibration (incorporating covariates for temperature, relative humidity and O₃ sensitivity) showed to yield acceptable sensor performance (R²=0.75-0.83, MAE=6-44 µg/m³), as observed in former studies [8,30,31,34,45]. In this study, we were not able to evaluate the precision of the considered sensor systems. We can, therefore, not conclude anything on network applications.

Regarding BC, both considered sensor systems showed good field performance (R²=0.82-0.83, MAE=0.2-0.3 µg/m³), while we should mention that BCmeter can yet not be applied in mobile applications (wired power and wifi connectivity). The measurement principle of light attenuation on filterstrips has proven to be a robust methodology to measure black carbon in the past [12,46,47,48,49,50,51] and can be minimized to portable and lower-cost instruments. Moreover, the spatial BC exposure variability, measured by the Observair in the mobile field test was in good agreement with the Aethlabs MA200 measurements (Supplementary S12). In general, all sensor systems showed a good horizontal accuracy (<10m) with no vibration impacts on the sensor readings for all pollutants during the mobile field test.

5. Conclusions

This study evaluated the fit-for-purpose of commercially available portable sensor systems for dynamic exposure assessments in urban environments. We evaluated 10 sensor systems, measuring PM, NO₂ and/or BC, in both laboratory and real-world conditions. Besides quantitative performance assessments, qualitative experience on their portability, data transmission/storage and user friendliness were obtained throughout the experiments. Autonomous operation with internal GPS (no reliance on app connectivity) and data storage redundancy (SD storage besides cloud or app transmission) for example showed to be valuable assets in terms of data coverage. Results of the considered sensor systems indicate that out-of-the-box performance is relatively good for PM and BC, but maturity of the tested NO₂ sensors is still low and additional effort is needed in terms of signal noise and calibration. Multivariate calibration under field conditions showed promising performance for real-world applications. Future directions for PM and BC should focus on applicability (pollutant gradients in urban environments), added-value and user-friendliness (day-to-day use) of real-world applications, while for NO₂ research-proven noise reduction and calibration procedures [8,30,31,34,52,53,54] should be implemented in commercial instruments to increase the level of maturity in the market. This work shows that commercially available portable sensor systems have reached a good maturity level for PM and BC, while more work is needed for NO₂. More accurate and dynamic exposure assessments in contemporary urban environments are crucial to study real-world exposure of individuals and the impact on potential health endpoints [14,55,56,57,58,59,60,61]. This research will be boosted by the greater availability of mobile monitoring systems capable of quantifying urban pollutant gradients.