Multi-Site Multi-Pollutant Air Quality Data Modeling

1. Introduction

The issue of air quality is not just a health concern but also a pressing issue of sustainable development. With the rapid development of the global economy, the challenge of air pollution has become increasingly apparent. The issues of air quality are complicated and have received increasing attention in China. In most cities, multiple monitoring stations are distributed at different locations to report real-time air quality indices. Typically, the levels of air pollutants are periodically recorded by multiple stations (for example, the data used in this paper are recorded every hour). It means that at every time stamp one air quality matrix with the shape of $station\times feature$ can be collected by all the stations, where features include NO₂, CO, PM 2.5, PM 10 etc. Based on this air quality data matrix, an air quality index (AQI) can be calculated to inform public the air quality at present. However, the general public are more interested in prediction of future air quality rather than the real-time reporting. Not only such prediction can benefit people’s daily life activities (for example, making a travel plan or avoiding routes of poor air quality) and improve their health by wearing masks to reduce exposure to air pollution, but it also provides policy implications for the government. There are a large number of works for air quality prediction in the literature [1,2,3]. Most of them are based on the temporal dependency between future states and historical data, such as time series models [2,4,5] and deep neural networks [1,6,7,8]. However, there are several limitations to the existing methods. First, many models [2,6,7] take the air quality prediction as a single-pollutant regression problem, for example, focusing on only the particulate matter PM 2.5. To predict the level of another pollutant, e.g., the carbon monoxide CO, a different model needs to be trained. Second, to improve the performance for prediction, some methods [5,9] choose to incorporate extra knowledge, such as the weather forecasting results [9] or the traffic data [5]. In practice, it is not convenient to collect the extra information and synchronize them with the air quality data. When data from multiple stations are available, the geographical correlations among these stations are expected to be useful for air quality forecasting [5,9]. However, most existing methods can only deal with the data from one station at a time such that the spatial correlations among multiple stations are ignored [1,2,7] or partially considered [5,9]. Last but not least, the air quality data usually represent high nonstationarity as shown in Figure 2, with the mean of data changing over time, which makes the modeling problem more difficult. Therefore, learning the underlying spatio–temporal features from a nonstationary process is particularly crucial for prediction. We propose to solve the aforementioned problems using a nonstationary diffusion graph convolutional LSTM (long short term memory). In detail, the spatio–temporal characteristics of air quality data from multi-sites motivate us to use the diffusion convolutional LSTM network [10]. The diffusion convolution [11] captures the spatial dependency using bidirectional random walks on the meteorological monitoring sites graph $\mathcal{G}=(V,E,A)$ as shown in Figure 1 (b). In addition, we add a de-trending step to the diffusion convolutional LSTM to accommodate the nonstationarity of the data. We name the proposed model as the long-short de-trending graph convolutional network (LS-deGCN). In time series analysis, the de-trending step is usually implemented by a differencing procedure ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ [12,13]. As a result, the input of the proposed model involves both ${\mathbf{x}}_t$ and ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$.

Motivated by the vast applications of LSTM in the area of natural language processing (NLP), we propose two variants of the LS-deGCN, as shown in Figure 3. At every time stamp, there is one $station\times feature$ matrix of data $\mathbf{x}\in {\mathbb{R}}^{M\times N}$ observed, where M is the number of stations and N is the number of features. Therefore, the final dataset is a 3-dimensional tensor $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$ by stacking all $\mathbf{x}$’s along time, where T is the number of time stamps. Given a window length, for example 3, which is a tuning parameter in our model, we can slice the samples along the third dimension T. For ease of exposition, we omit the first two dimensions and denote $\mathcal{X}[0:3]$$=\mathcal{X}[:,:,0:3]$. The sliced samples are in the form of ${\mathbf{x}}_1=\mathcal{X}[0:3],{\mathbf{x}}_2=\mathcal{X}[1:4]$, etc. We propose two different ways of defining the target $\mathbf{y}$, which correspond to the two variants of our models. One is a sequence-to-frame model; that is, ${\mathbf{y}}_1=\mathcal{X}\left[4\right],{\mathbf{y}}_2=\mathcal{X}\left[5\right]$, etc; and the other is a sequence-to-sequence model; that is, ${\mathbf{y}}_1=\mathcal{X}[1:4],{\mathbf{y}}_1=\mathcal{X}[2:5]$, etc.

The contributions of this paper are threefold:

• We modify the traditional LSTM by adding a de-trending operation for nonstationary data;
• We propose to use the diffusion graph convolution to extract the spatial correlations in the air quality data from multi-sites;
• We propose two different models based on the LS-deGCN to predict air quality at multi-sites and evaluate them on the air quality data in the city of Chengdu and the other data from seven major cities.

The rest of the paper is structured as follows. Section 2 presents a brief review of related works. In Section 3, we introduce the proposed LS-deGCN and its two variants:the sequence-to-frame model and the sequence-to-sequence model. The experimental results are shown in Section 4. Section 5 concludes this paper with some remarks.

2. Literature Review

Traditionally, meteorologists often make air quality prediction based on their empirical knowledge of meteorology. With the development of statistics and machine learning, data-driven methods for air quality prediction are becoming increasingly popular nowadays, which can be typically divided into statistical approaches [2,14,15,16,17] and deep learning approaches [6,7,9,10,18,19]. Some existing deep learning models treat the air quality forecasting as a single pollutant regression problem, and thus they only predict one pollutant at a time [2,6,20]. As a result, separate models need to be trained for different pollutants if all of the pollutants are of interest, where each model only focuses on one pollutant. Zhang et al. [2] conducted a statistical analysis of the PM 2.5 data in years 2013–2016 from the city Beijing based on a flexible nonstationary hierarchical Bayesian model. Mukhopadhyay and Sahu [17] proposed a Bayesian spatio–temporal model to estimate the long-term exposure to air pollution levels in England. Since the air quality records are typically monitored over time, Ghaemi et al. [15] designed an LaSVM-based online algorithm to deal with the streaming of the air quality data. Along another direction, the Granger causality has been proposed to analyze the correlations among the air pollution sequences from different monitoring stations [4,5,9]. Suppose that the sequence of air pollutant records from one station is denoted by ${\mathbf{x}}_t$, and the sequence of a factor (such as the geographical correlation) from another station by ${\mathbf{y}}_t$, then the mathematical representation of the Granger causality is given by

$${\mathbf{x}}_t=\sum _{k=1}^K{a}^k{\mathbf{x}}_{t-k}+\sum _{k=1}^K{b}^k{\mathbf{y}}_{t-k}+{ϵ}_t,$$

(1)

where $a^k$ is a weight indicating how the width of time window k affects the future evolution, $b^k$ represents the correspondent weight for ${\mathbf{x}}_t$ and ${\mathbf{y}}_t$, and ${ϵ}_t$ is a residual for time series ${\mathbf{x}}_t$. If $a^k\ne 0$ and $b^k=0$, it means that the sequence ${\mathbf{x}}_t$ is caused by its own history. Wang and Song [9] combined the Granger causality with deep learning models and there are also some connections between the Granger causality and LSTM [21].

The deep learning, more specifically, the recurrent neural network (RNN) and LSTM [21] have achieved vast success in the area of NLP [22]. Due to their capability in modeling sequence data, the RNN and LSTM demonstrate good performance in air quality prediction as well [1,8,23]. Guo et al. [23] proposed a multi-variable LSTM based on both temporal and variable level attention mechanisms, which was used to predict the PM 2.5 level in Beijing. Fan et al. [8] also used the LSTM as a framework to predict air quality in Beijing based on the air pollution and meteorological information. In contrast to [23], the data used in [8] are collected from multiple stations, while data from different stations are analyzed separately by ignoring the spatial correlation. In big cities, there are usually more than one stations deployed to monitor the air pollutants and meteorological information. The correlations among readings from different stations are highly informative in forecasting future air quality and thus the spatial information should be incorporated. Xu et al. [24] proposed a multi-scale 3-dimensional tensor decomposition algorithm to handle the spatio–temporal correlation in climate modeling. In deep learning, the convolutional neural network (CNN) has advantages in extracting spatial features, while the RNN has superior performance in processing sequence data. Therefore, it is expected to achieve more accurate prediction by combining convolution and RNN in the analysis of spatial and temporal data. Huang and Kuo [6] proposed to stack a CNN over LSTM to predict the level of PM 2.5. However, this modification leads to a non-time-series model, which may cause power loss in quantifying the sequential air quality data. Wilson et al. [25] proposed a deep learning approach based on the graphical CNN for weather forecasting, where a weighted graph, aiming to capture the spatial correlation, is calculated based on the original features.

In this paper, we propose to model the air quality data from multiple stations with a LS-deGCN, which replaces the fully connected layer in the classical LSTM with graph diffusion convolution. The LS-deGCN architecture can be well adapted to the spatio–temporal data, which was first proposed to analyze the 2-dimensional radar echo map [10]. In addition, our model can accommodate the nonstationarity of the air quality data. Compared with the work of Wilson et al. [25], our method is easier to be implemented and trained. A similar diffusion convolutional recursive neural network was proposed in [26] to model the traffic flow.

3. Proposed Models

3.1. Problem and Research Gap

The LSTM network is generally designed to deal with sequential data [21], which has achieved great success in NLP [22] and video analysis [27]. The LSTM can capture both the long and short contextual dependency of the data via different types of gates. In the classical LSTM, the well-designed gates, for example, the input gate and the forget gate, make the network very powerful to model the temporal correlations of the sequential data.

For the problem of air quality prediction, the data are hourly recorded by multiple stations. At each time stamp, the observed data can be represented by a $station\times feature$ matrix $\mathbf{x}\in {\mathbb{R}}^{M\times N}$, where M is the number of stations and N is the number of features. The features refer to air pollutants (e.g., CO₂, PM 2.5) and meteorological parameters (e.g., air pressure, air temperature, and air humidity). As shown in Figure 1, the air quality data of the city Chengdu involve 9 monitoring stations and 9 features. The entire data thus can be treated as a 3-dimensional tensor $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$, with respect to the three axes $station\times feature\times time$. The third dimension T corresponds to the number of time stamps, which indicates the sequential nature of the data $\mathcal{X}$.

The air quality data in Chengdu have a typical spatio–temporal pattern, as the data from different stations show strong spatial correlations. Figure 1 (a) are the locations of different monitoring stations. According to the pairwise geographic distances between different stations, we compute the undirected graph $\mathcal{G}=(V,E,A)$ with thresholded Gaussian kernel [28], $A_{ij}=exp(-\frac{\parallel v_i-v_j\parallel }{{\sigma }^2})$, as shown in Figure 1 (b), where $A_{ij}$ is an element of the adjancent matrix A, $v_i$ and $v_j$ are the $i,j$-th nodes of $\mathcal{G}$. Figure 1 (C) exhibits the one-week readings of two types of pollutants, i.e., CO and PM 2.5 from station 4 and station 5 respectively. There are clear shifts from station 4 to station 5 in the records of CO and PM 2.5, where the patterns are similar. The area where station 5 is located appears to be more polluted than the area of station 4, and such information is useful for government policy making in different districts. The similar oscillating patterns in the records of the two stations imply that the rows and columns in the matrix $station\times feature$ are correlated and further series correlation over time can also be observed. However, there seems no obvious daily periodicity or “weekends/holidays” effect from the data. Although the classical LSTM possesses powerful capability in modeling time series data, it may not be suitable to capture such type of spatial correlation in the data [8].

3.2. Nonstationary Diffusion Convolutional LSTM

We propose to model the nonstationary $station\times feature\times time$ data with the de-trending diffusion convolutional LSTM (LS-deGCN), which aims to incorporate both the spatio–temporal correlations [10] and nonstationarity into our model. In contrast to the classical LSTM where all the gates are implemented by fully connected neural networks, the proposed model replaces them with diffusion operations [11]. In addition, the differences of the input signals ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ and corresponding hidden signals ${\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}$ are input for de-trending as follows,

$$\begin{array}{cc}\hfill f_t& =\sigma \left(W_f\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{df}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+W_{cf}\circ {\mathbf{c}}_{t-1}+b_f\right)\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill i_t& =\sigma \left(W_i\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{di}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+W_{ci}\circ {\mathbf{c}}_{t-1}+b_i\right)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\mathbf{c}}_t& =f_t\circ {\mathbf{c}}_{t-1}+i_t\circ tanh\left(W_c\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{dc}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+b_c\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\mathbf{o}}_t& =\sigma \left(W_o\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{do}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+W_{co}\circ {\mathbf{c}}_t+b_o\right)\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\mathbf{h}}_t& ={\mathbf{o}}_t\circ tanh\left({\mathbf{c}}_t\right),\hfill \end{array}$$

(6)

where ∗ and ∘ represent the diffusion convolution and Hadamard product respectively, $[\mathbf{a},\mathbf{b}]$ is concatenation of vectors $\mathbf{a}$ and $\mathbf{b}$, $\sigma (·)$ is the $sigmoid$ function, and $tanh(·)$ is the hyperbolic tangent function. Eqs (2), (3) and (5) correspond to the implementations of the forget gate, input gate, and output gate. Eq (4) is the updating mechanism of the cell state ${\mathbf{c}}_t$, and the hidden state ${\mathbf{h}}_t$ is updated by Eq (6). Note that $\{W_f,W_{df},W_{cf},b_f\}$ are the weights (corresponding to the feature, de-trending term and hidden state) and bias of the forget gate, and $\{W_i,W_{di},W_{ci},b_i\}$ and $\{W_o,W_{do},W_{co},b_o\}$ are the counterparts of the input gate and the output gate respectively. Furthermore, $\{W_c,W_{dc},b_c\}$ are the weights (corresponding to the feature and de-trending term) and bias of the hidden state.

The diffusion convolution [11] is defined as follows,

$$\widehat{\mathbf{x}}=\sum _{m=1}^M\sum _{k=0}^{K-1}(w_{k1}{\left(D_O^{-1}A\right)}^k+w_{k2}{\left(D_I^{-1}A\right)}^k)\mathbf{x}$$

(7)

where $\mathbf{x}$ is an input matrix and $\widehat{\mathbf{x}}$ is the corresponding output. A is an adjacent matrix of the M stations as shown in Figure 1 (b), and $D_O$ and $D_I$ are the out-degree and in-degree diagonal matrix of A.

The frame differencing term ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ is governed by the corresponding coefficient matrices. In Eq (5), we observe that not only does the current output ${\mathbf{o}}_t$ depend on the current input ${\mathbf{x}}_t$, the current cell state ${\mathbf{c}}_t$, and the previous hidden state ${\mathbf{h}}_{t-1}$, but it also requires the input of the differencing signal ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$. We also concatenate the hidden differencing signal ${\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}$ with the input differencing ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$.

The main differences between the classical LSTM and our LS-deGCN lie in the diffusion graph convolutional operations and the extra de-trending items ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ and ${\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}$. If the graph convolution in Eqs (2)–(6) are substituted with inner products and set ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}=0,{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}=0$, then the LS-deGCN reduces to the classical LSTM. The great success achieved by the graph convolutional neural network is largely owed to the graph convolution which can effectively capture the spatial information of data. This is also the rationale underlying our LS-deGCN to characterize the spatial correlations.

As discussed earlier, the air quality data are typically collected from multiple stations over time, which represent both the spatial and temporal characteristics. The spatial correlations among different stations depend on many complicated factors such as geographical distances, directions and meteorological features, which are very important for air quality prediction [4,5,9]. In the existing works, the spatial correlations are typically inferred manually by resorting to some complicated statistical processes. In this paper, we propose to model the spatial correlations with diffusion graph convolution. In other words, the spatial correlations among different stations can be extracted by the LS-deGCN, which is the most notable difference between our method and the existing models for air quality prediction. This distinctive feature makes our model coherent in the deep learning framework and more powerful for air quality forecasting.

3.3. Two new models

According to the scheme of generating samples and targets as shown in Figure 3, we develop two variants of our modeling framework. To help further elaboration, we first introduce some necessary notation. We first select ${\mathsf{\Delta }}_t$, the width of a slicing window or the length of the samples, which is an analogy to the sentence length in NLP. The value of ${\mathsf{\Delta }}_t$ represents how many historical sample observations are used for prediction. Let l denote the time lag from a sample to the target; that is, our goal is to make a prediction of air quality for l hours later. We then propose the following two modeling frameworks.

In the sequence-to-frame (seq2frame) model, we take samples to be slices of the 3-dimensional tensor $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$ along the dimension T. The sample length is ${\mathsf{\Delta }}_t$, the width of the slicing window. The target is taken to be the single frame after the corresponding samples. Since the time lag from a sample to its target is l, the training and testing samples can be generated as follows,

$$\begin{array}{cc}\hfill {\mathbf{x}}_i& =\mathcal{X}[i-1:{\mathsf{\Delta }}_t+i-1],\hfill \\ \hfill {\mathbf{y}}_i& =\mathcal{X}[{\mathsf{\Delta }}_t+i+l-1].\hfill \end{array}$$

Figure 3 (a) illustrates the way to generate ${\mathbf{x}}_i$ and ${\mathbf{y}}_i$ in the seq2frame model, with ${\mathsf{\Delta }}_t=5$ and time lag $l=1$. As a result, we can obtain the first sample and its target, i.e., ${\mathbf{x}}_1=\mathcal{X}[0:5]$, and ${\mathbf{y}}_1=\mathcal{X}\left[6\right]$.

The sequence-to-sequence (seq2seq) model is motivated by the seq2seq model in NLP [22], under which the samples and targets are generated as in Figure 3 (b) with both the sample and target of length ${\mathsf{\Delta }}_t$ slices. We use the same way of slicing samples $\mathbf{x}$ as that for the seq2frame scheme and define a time l-shifted sequence after $\mathbf{x}$ to be the target $\mathbf{y}$. This data generating scheme leads to

$$\begin{array}{cc}\hfill {\mathbf{x}}_i& =\mathcal{X}[i-1:{\mathsf{\Delta }}_t+i-1],\hfill \\ \hfill {\mathbf{y}}_i& =\mathcal{X}[i+l-1:{\mathsf{\Delta }}_t+i+l-1].\hfill \end{array}$$

As illustrated in Figure 3 (b), with ${\mathsf{\Delta }}_t=5$ and time lag $l=1$, the first sample is ${\mathbf{x}}_1=\mathcal{X}[0:5]$ and its target is ${\mathbf{y}}_1=\mathcal{X}[1:6]$.

3.4. Selection of tuning parameters l and ${\mathsf{\Delta }}_t$

The time lag l and window width ${\mathsf{\Delta }}_t$ are critical tuning parameters for the proposed models. If the value of l is too small, the overlapping of the information would be too much. In contrast, a larger value of time lag l would make the prediction more difficult. Theoretically, the proposed model can predict the future air quality for any length of the window by setting l to be large enough. In practice, we explore the value of l from 1 hour to 48 hours, which means that we aim to predict the air quality till the day after tomorrow. Intuitively, the more past data are used, the better performance would be expected. In other words, the performance is usually monotonically increasing with the value of ${\mathsf{\Delta }}_t$.

4. Experiments and Results

4.1. Baselines

We choose the following three models as the baselines for comparisons.

• Linear regression: This is one of the most commonly used approaches to modeling the relationship between a dependent variable y and covariates $\mathbf{x}$.
• Support vector regression: Equipped with a radial basis kernel, it extends linear regression by controlling how much error in regression is acceptable.
• LSTM sequence-to-scalar (seq2scalar): Samples under this model are constructed in the same way as those under the seq2seq model. The difference is that we take the target y as one of the nine pollutants one by one; that is, we need to train nine separate models for the nine pollutants.

The three baseline methods are all one-pollutant regression procedures; that is, we need to train separate models on each individual pollutant at a time. Taking the linear regression as an example, to compare with the proposed model that predicts the whole $station\times feature$ map, we need to separately train nine linear regression models corresponding to the six air pollutants (NO₂, CO, SO₂, O₃, PM 2.5, and PM 10) and the three meteorological measurements.

We use the root mean squared error (RMSE), accuracy, and mean absolute error (MAE) as assessment criteria for prediction,

$$\begin{array}{ccc}\hfill \mathrm{RMSE}& =& \sqrt{\frac{{\sum }_{i=1}^n{({\widehat{\mathbf{y}}}_i-{\mathbf{y}}_i)}^2}{n}},\hfill \\ \hfill \mathrm{Accuracy}& =& 1-\frac{{\sum }_{i=1}^n|{\widehat{\mathbf{y}}}_i-{\mathbf{y}}_i|}{{\sum }_{i=1}^n{\mathbf{y}}_i},\hfill \\ \hfill \mathrm{MAE}& =& \frac{{\sum }_{i=1}^n|{\widehat{\mathbf{y}}}_i-{\mathbf{y}}_i|}{n},\hfill \end{array}$$

where ${\mathbf{y}}_i$ is the ground truth, ${\widehat{\mathbf{y}}}_i$ is the predicted value, and n is the size of the testing dataset.

4.2. Data description and preprocessing

The Chengdu dataset is composed of the air quality records from January 1, 2013 to December 31, 2016 from nine monitoring stations in the city of Chengdu. Because stations 3 and 9 were very close to each other and their records were exactly the same, we removed the redundancy by dropping the data from station 9. Moreover, we deleted the data from station 8 because over 40% readings for CO, SO₂ and O₃ were missing which mainly occurred between June 2014 to January 2016 due to the sensor dysfunction. As a result, we only used data from seven stations, and there are 35,064 instances for each station. Each air quality instance consists of the concentration of six air pollutants: NO₂, CO, SO₂, O₃, PM 2.5, PM 10, and three meteorological measurements including air pressure, air temperature, and air humidity. Therefore, the observed data are in the form of $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$, where $M=7$, $N=9$, and $T=\mathrm{35,064}$. We use the linear interpolation to fill in the missing values of $\mathcal{X}$ on the time domain for each column. For example, we can interpolate the missing ${\mathbf{x}}_i$ with the values of ${\mathbf{x}}_{i-1}$ and ${\mathbf{x}}_{i+1}$. After interpolation, we normalize all the data into the range of $[0,1]$ using the minmax normalization.

The training and testing samples for baselines are generated in the same way as those for the seq2frame model. The difference is how to extract the target. In the seq2frame architecture, the network receives one frame (matrix) as a target, while the targets of the baseline models are all scalars. Instead of using the whole frame as the target, we take a statistic (e.g., the mean) of the measurement of one pollutant within that frame as the target. Taking the linear regression as an example, we model the air quality prediction as a one-pollutant linear regression problem. Suppose that we fit a linear regression model to predict the PM 2.5 emission, then we can use the mean or the median of the PM 2.5 level within that frame as the target.

4.3. Training

After data normalization, we extracted the training samples and testing samples from the dataset $\mathcal{X}$. We set the window width to be ${\mathsf{\Delta }}_t=\{24,48\}$ hours to strike a balance between the length of contextual information and the LSTM model complexity. As a result, every sample is a consecutive record of air quality measurements for one day or two days. We partitioned the data into three sets as shown in Table 1. We constructed a two-layer LS-deGCN by stacking two one-layer LS-deGCNs.

Figure 4 (a) and (b) report the training and validation errors under the seq2frame and seq2seq models respectively. Both models converged in 10 epochs, while the patterns of their convergence are different. Under the seq2frame model, both the training error and validation error drop sharply within the first three epochs and then the gap between them gradually decreases. By contrast, the gap between the training error and validation error under the seq2seq model remains at a stable level even after 15 epochs. The different error patterns may be due to the different ways of extracting the target variable $\mathbf{y}$ under the two models, as shown in Figure 3.

4.4. Visualization of predictions

To visualize part of the predicted results, Figure 5 shows four $station\times feature$ air quality maps randomly selected from the testing dataset, as well as the corresponding predicted $station\times feature$ frames by the seq2seq model with $l=24$ and ${\mathsf{\Delta }}_t=48$. For each square of the $station\times feature$ matrix, the dark blue color represents a larger value and the light blue color indicates a smaller value.

The far right-end three columns (corresponding to the three meteorological measurements: air pressure, air temperature, and air humidity) of the ground truth map are almost identical across seven stations. In contrast, the levels of air pollutants recorded by different stations are quite different. For example, the levels of O₃ (the third column) reported at different locations show very different patterns, and in the third frame station 2 and 5 reported a much higher value than other stations. The proposed model can make a accurate prediction of O₃ for each station. We also observe that the values of air pressure in the third frame are much lower than the other three frames, and the level of suspended particulate matter, PM 10 and PM 2.5, in the last frame are serious, with the level of PM 10 from station 1 and 6 being the top 2 highest. These trends are all correctly predicted by our proposed seq2seq model. From the visualization results, we conclude that (1) the patterns of the nine air quality measurements are different from one another; and (2) the proposed model can make an accurate prediction based on all of the nine measurements from seven stations.

To examine the proposed air quality model across different cities, we conducted another experiment with seven stations from seven major cities in China, including Beijing, Shanghai, Chengdu, Wuhan, Guangzhou, Xi’an, and Nanjing. The data were collected daily (instead of hourly) on seven pollutants (AQI, PM 2.5, PM 10, SO₂, CO, NO₂, and O₃) from December 2, 2013 to February 29, 2020 for each city. Figure 6 shows that the performance of air quality prediction for seven cities is not as good as that for seven stations from the same city of Chengdu. One possible reason is that the spatial correlation among seven stations in Chengdu is much stronger than the seven cities that are far away from each other, and thus the proposed LS-deGCN is more effective.

4.5. Evaluation with three metrics

RMSE. As discussed in Section 3.3, the proposed model can predict the future level of air pollution from 1 hour to 48 hours by varying l from 1 to 48. Moreover, the data used for prediction can be a collection of samples in the last day or last several days, for which ${\mathsf{\Delta }}_t$ is used to control the length of samples. In other words, we can vary the parameter ${\mathsf{\Delta }}_t$ to obtain different training and testing datasets for different experiments. We explore six experiments by combinations of $l=\{1,24,48\}$ and ${\mathsf{\Delta }}_t=\{24,48\}$. Therefore, we utilize the data collected from yesterday and the last two days to predict the air quality for one hour later, one day later, and two days later, respectively.

Table 2 displays the RMSEs from the testing set for $l=\{1,24,48\}$ with respect to ${\mathsf{\Delta }}_t=\{24,48\}$. The RMSEs of the nonstationary LS-deGCN seq2frame and seq2seq models can be calculated directly based on the $station\times feature$ matrix. However, for the other three models, we need to conduct training and testing for each pollutant separately, and then calculate the average RMSE over all the pollutants. It is clear that our proposed models demonstrate significant improvements compared with the baseline methods and, in particular, the nonstationary LS-deGCN seq2seq model achieves the best performance under all the six scenarios. The LSTM-based methods outperform the traditional linear regression and support vector regression. Moreover, the RMSEs of our models increase as the time lag l increases and decrease as ${\mathsf{\Delta }}_t$ takes a larger value. Intuitively, the prediction of air quality for two days later would be more challenging than that for one hour later, and the prediction would improve with more samples.

Accuracy. Table 3 compares the accuracy of air quality prediction among the five methods for combinations of $l=\{1,24,48\}$ and ${\mathsf{\Delta }}_t=\{24,48\}$. The proposed nonstationary LS-deGCN seq2frame and seq2seq models yield much higher accuracy than the others, and the nonstationary LS-deGCN seq2seq model achieves the highest accuracy among all.

MAE. Table 4 shows the MAE of all the methods for the six scenarios with $l=\{1,24,48\}$ and ${\mathsf{\Delta }}_t=\{24,48\}$. In those experiments, the two proposed models, the nonstationary LS-deGCN seq2frame and seq2seq models, report much lower MAE than the others, and the nonstationary LS-deGCN seq2seq model outperforms all the rest of competitors.

For the three metrics, we also observe that the scenarios with ${\mathsf{\Delta }}_t=48$ outperform those with ${\mathsf{\Delta }}_t=24$, especially for the LSTM-based methods (i.e., LSTM seq2scalar, nonstationary LS-deGCN seq2frame and seq2seq). This is expected as prediction of air quality with the historical data of the last two days is a better strategy than that of only utilizing the data of yesterday.

For the results of RMSE and MAE in Table 2 and Table 4, we observe that the performances continue to decrease when the value of l increases. However, for the accuracy results in Table 3, we observe that the performance does not monotonically decrease with an increasing value of l for our models. In contrast, the accuracy achieves a peak at $l=24$, and the performance deteriorates for both $l=1$ and $l=48$. This indicates that the three evaluation metrics do not exactly match with each other. As discussed earlier, the larger the time lag l, the more difficult it is to make a prediction. However, if l is too small, the overlapped information under the nonstationary LS-deGCN seq2seq model would be large.

5. Conclusions

In this paper, we train a non-stationary LS-deGCN to fit the spatial-temporal data, a multi-station air pollution dataset of Chengdu from January 1, 2013 to December 31, 2016. On one hand, the de-trending long-short term structure of LS-deGCN can capture the non-stationary temporal dependency between the predicted values and the historical records. On the other hand, the diffusion graph convolution on the $station\times features$ grid gains more power to extract the spatial dependency among stations than a fully connected network. According to the scheme of extracting samples and targets, we propose two sub-models, the non-stationary LS-deGCN seq2frame and non-stationary LS-deGCN seq2seq. The target of the former model is a frame while the later is a l-shifted sequence. We evaluate both models based on the RMSE, accuracy, and MAE and demonstrate their outstanding performances in comparison with existing works. However, air quality prediction involves many other factors such as geographical information and traffic information. In the future work, we will incorporate other data and integrate them into the framework of the non-stationary LS-deGCN.