FCP-Former: Enhancing Long-Term Multivariate Time Series Forecasting with Frequency Compensation

1. Introduction

Time series forecasting constitutes a statistical approach aimed at predicting future observations based on historical temporal data. This methodology has demonstrated extensive applicability across a broad spectrum of domains, including but not limited to meteorology[1,2], healthcare analytics[3,4], intelligent transportation systems[5,6] , electrical load forecasting[7,8], and financial risk assessment[9,10]. In recent years, recurrent neural network (RNN)-based architectures have been extensively employed for modeling time series data due to their capacity to learn temporal dependencies[11,12]. While these methods have yielded considerable empirical success [13,14], they are inherently constrained by several limitations, most notably the issues of vanishing and exploding gradients. These challenges significantly hinder the ability of RNNs to effectively model long-range dependencies within sequential data, thereby limiting their performance in scenarios requiring long-term forecasting accuracy.

After achieving great success in computer vision [15,16,17,18] and natural language processing [19,20,21,22], the transformer [23] model was introduced to time series forecasting to directly model the relationships between any two time steps in a sequence. Due to its powerful attention mechanism, transformer overcomes the gradient vanishing and gradient exploding problems that still trouble RNN and LSTM(Long Short-Term Memory)-type methods, making it a popular research topic in the field of time series forecasting.

Based on the token granularity fed into the attention mechanism in the time domain, existing Transformer-based research can be roughly divided into patch-wise models and point-wise models. A patch is a basic module formed by concatenating multiple temporally contiguous time-series data points. This enables the model to treat a patch as a token instead of treating each timestep as a token, significantly reducing the computational time. Based on different treatments of the variates, patch-wise models can be further divided into channel-independent strategy models and channel-dependent strategy models. Typical channel-independent strategy models include PatchTST [24], while channel-dependent strategy models include iTransformer[31] ,TimeXer[32] and Crossformer[33]. In contrast, point-wise models treat each time step and its corresponding variates as a token, which gives them a stronger ability to capture internal temporal variations. Typical point-wise models include FEDformer [25], Informer [26] and Autoformer [27]. However, due to their high computational complexity, it is challenging for these models to capture long-term dependencies between time series data.

For patch-wise models, patching the time series data can effectively reduce computational resource consumption. However, patch-wise methods embed each patch into a coarse token through a temporal linear projection, which leads to their inability to fully utilize the data within the patch, potentially compromising the accuracy of the final prediction.

Inspired by FEDformer [25] which utilizes frequency domain transformations, we recognize that some information in time series data may not be sufficiently captured in the time domain but can be effectively revealed in the frequency domain. For instance, periodic signals exhibit this characteristic.. As illustrated in Figure 1, in the time domain, data points are arranged in chronological order, with each data point representing the observation at a specific point in time. In the frequency domain, the data is decomposed into different frequency components, with each frequency component representing the extent of a particular periodicity in the signal. This representation facilitates the identification of periodicity and underlying trends in the data.

To address the loss caused by the model's inability to fully utilize the information of each piece of data within the patch, we propose a method that adds corresponding frequency domain information to the patched data, compensating for the information loss caused by patching, and propose an optimized PatchTST[24] called Frequency Compensation Patch-wise transFormer(FCP-Former). The main contributions of this paper are summarized as follows:

• We propose a frequency compensation block that adds corresponding frequency domain information to the patched data via frequency-domain representation learning, compensating for intra-patch information loss.
• We use the frequency compensation block to optimize PatchTST model, called FCP-Former, which better captures the periodic and trend changes in time series data.
• We conducted multivariate time series prediction experiments on eigth publicly available multivariate time series datasets. The proposed FCP-Former exhibits better comprehensive performance compared with the state-of-the-art methods.

2. Preliminaries and Related Work

2.1. Problem Definition

Time series data is a set of data arranged in chronological order. This type of data is typically collected at specific time points, and there is a temporal dependence between the data points. In time series forecasting, future events are predicted by utilizing these time-ordered data. The historical data can be defined as $X_t=\{{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_{L-1},{\mathbf{x}}_L\}\in {\mathbb{R}}^{L\times D}$ and the predicted data can be defined as ${\widehat{X}}_t=\{{\widehat{\mathbf{x}}}_{L+1},{\widehat{\mathbf{x}}}_{L+2},\dots ,{\widehat{\mathbf{x}}}_{L+T-1},{\widehat{\mathbf{x}}}_{L+T}\}\in {\mathbb{R}}^{T\times D}$. where D is the number of variables, L is the length of historical data, T is the length of predicted data. The concept of time series forecasting can be expressed as:

$$\begin{array}{c}{\widehat{\mathrm{X}}}_{\mathrm{t}}=f\left({\mathrm{X}}_{\mathrm{t}}\right)+ϵ,\end{array}$$

(1)

where ${\widehat{X}}_t$ is the predicted value, $f$ is the forecasting function, $X_t$ are the historical values and $ϵ$ is the forecasting error.

2.2. Transformer-Based Time Series Forecaster

With the great success made in the field of natural language processing and computer vision, Transformer has gained the attention of researchers in the field of time series forecasting due to its powerful ability to capture long-term temporal dependencies and complex multivariate correlations. We briefly review several key variants below. Informer [26] addresses the high computational complexity of transformers in time series forecasting by proposing a sparse self-attention mechanism. FEDformer[25] enhances the transformer model's ability to capture global features of time series data by combining the transformer model with seasonal trend decomposition, while retaining key frequency information of the time series data through Fourier and wavelet transforms. PatchTST[24] improves the transformer’s ability to capture historical dependencies by using a channel-independent strategy to patch the time series data, reducing computational overhead while maintaining the ability to model long-range dependencies. Crossformer[33] enhances the transformer’s ability to handle multivariate time series forecasting tasks through dimension-segment-wise embedding and a two-stage attention mechanism. Npformer[28] introduces an innovative multi-scale segmented Fourier attention mechanism to more effectively capture dependencies. TimeXer[32] enhances the Transformer model's prediction accuracy by incorporating exogenous variables. iTransformer[31] applies the Transformer's attention mechanism along the variate dimension instead of the time dimension.

Most of these transformer-based models either focus on the attention mechanism, designing new attention mechanisms to reduce the complexity of the original attention mechanism, or process the time series data itself to better leverage the transformer, thus achieving better performance on forecasting, especially when the prediction length is long. However, these patch-wise transformer methods face a common problem — as shown in Figure 2, compared to point-wise methods, the patch-wise approach, where the model treats a patch as a single token, cannot fully utilize each piece of data, which results in information loss within the patch. TimeXer introduces exogenous variables to address this issue, however, the feature extracted from the time domain remains inherently limited. In contrast to TimeXer, we leverage the complementary nature of frequency-domain information to time-domain data. Our proposed frequency compensation block extracts features from the frequency domain, effectively overcoming the limitations of relying solely on time-domain feature extraction. This enables the model to better capture the periodic and trend characteristics of time series data.

2.3. Time Series Forecasting with Time-Frequency Analysis

The Fourier transform serves as a bridge for converting signals between the time and frequency domains, with the discrete Fourier transform (DFT) and discrete wavelet transform (DWT) commonly used tools for time-frequency analysis. Current mainstream time-frequency analysis methods can be categorized into two types. The first type involves transforming time-domain data into the corresponding Fourier spectrum, analyzing the Fourier spectrum to extract frequency-domain-based features, and then using inverse transformations to convert the data back to the time domain to obtain prediction results. Typical examples include FreTS [29], FITS [34], and SparseTSF [35]. In contrast, the second type simultaneously extracts features from both the time and frequency domains of time series data, with the extracted features then concatenated at the network output to produce the prediction result. A typical example is FEDformer [25]. The method proposed in this paper primarily addresses the issue of data loss within patches in patch-wise models. Since the first type of time-frequency analysis typically demands relatively low resource overhead, the proposed FCP-Former adopts this approach.

3. Method

As illustrated in Figure 3, our proposed FCP-Former includes the following components: patching, embedding, projection, encoder, and the frequency compensation block. Lu Han et al.[30] had demonstrated through extensive experiments that the prediction method using channel-independent strategies typically achieves better prediction results than the method using channel-dependent strategies. Therefore, FCP-Former, like PatchTST[24], adopts the channel-independent strategy. However, FCP-Former applies the frequency compensation block to process the patched data before encoding, adding corresponding frequency features to compensate for intra-patch information loss. While NPformer [31] designs a multi-scale segmented Fourier attention mechanism, iTransformer has demonstrated that the standard attention mechanism can also yield excellent results. Therefore, instead of modifying the attention mechanism, FCP-Former focuses on enriching the information within each patch.

3.1. Model Structure

FCP-Former consists of a patching module, a Frequency Compensation Block, and an encoder.

Patching: By adopting a channel-independent strategy, we divide the original time series data into D channels based on the data dimension D, and perform patching separately for each channel. The time series data in each channel can be represented as $X_t^i=\{x_1^i,x_2^i,\dots ,x_{L-1}^i,x_L^i\}\in {\mathbb{R}}^{1\times L}$, where i represents the i-th channel among the D channels. Let the patch length be P, the patch step size be S, and the length of the time series data in the channel be L. The number of patches, N, can then be calculated as follows:

$$N=⌈{\displaystyle \frac{L-P}{S}}⌉+1,$$

(2)

padding is applied at the end, meaning that when the last patch extends beyond the end of the series, the remaining positions within that patch are filled using the last observed value $x_L^i$ in the channel, ensuring that each patch has a consistent size. After patching, the original time series in each channel is transformed into a sequence of patches $P_n^i=\{p_1^i,p_2^i,\dots ,p_{N-1}^i,p_N^i\}\in {\mathbb{R}}^{P\times N}$.

Frequency Compensation Block: The core function of the frequency compensation block is to perform representation learning on each patch in the frequency domain, using the features of the patch in the frequency domain as a supplement to the information that is overlooked for each time step within the patch. To achieve this, frequency compensation block first applies a Fast Fourier Transform (FFT) to the patch, converting the data within the patch to a frequency domain representation. Then, it performs representation learning in the frequency domain. Finally, an inverse Fourier transform is applied to convert the data back into a time-domain representation. The data in the processed patch represents a transformed version of the original data, enriched by the frequency domain representation learning, rather than the completely raw data. It also includes the frequency characteristics of the patch, which serves as a compensation for the information that is overlooked within the patch. We will analyze the frequency compensation block in detail in the next section.

Encoder: We use a vanilla Transformer encoder to map the patches processed by frequency compensation block into the latent representations. Each patch is embedded into a latent space of dimension D by using a learnable linear projection matrix ${\mathrm{W}}_p\in {\mathbb{R}}^{D\times P}$ and position encoding ${\mathrm{W}}_{\mathrm{p}\mathrm{o}\mathrm{s}}\in {\mathbb{R}}^{D\times N}$, which serves as the input to the encoder. Embedding process can be simply formulated as follows:

$$FC{P}_n^i=FCBlock\left(P_n^i\right),$$

(3)

$${IN}_d^i={\mathrm{W}}_p{FCP}_n^i+W_{pos},$$

(4)

where $FCBlock$ is the frequency compensation block, $FC{P}_n^i$ is the result obtained after applying frequency compensation block to each patch, and ${IN}_d^i\in {\mathbb{R}}^{D\times N}$ is the embedded result used as the input of the encoder. Then the multi-head attention will transform them into query matrices $Q_h^i\in$, key matrices $K_h^i$ and value matrices $V_h^i$. The attention output $OU{T}_h^i\in {\mathbb{R}}^{D\times N}$ is ultimately obtained through scaled dot product. Attention process can be simply formulated as follows:

$$Q_h^i={\left(I{N}_d^i\right)}^T{W}_h^Q,$$

(5)

$$K_h^i={\left(I{N}_d^i\right)}^T{W}_h^K,$$

(6)

$$V_h^i={\left(I{N}_d^i\right)}^T{W}_h^V,$$

(7)

$${\left(OU{T}_h^i\right)}^T=\mathrm{Attention}\left(Q_h^i,K_h^i,V_h^i\right)=\mathrm{Softmax}\left({\displaystyle \frac{Q_h^i{K}_h^{i^T}}{\sqrt{d_k}}}\right)V_h^i,$$

(8)

where $W_h^Q,W_h^K\in {\mathbb{R}}^{D\times d_k}$ and $W_h^V\in {\mathbb{R}}^{D\times D}$. After passing through the BatchNorm layers and feed forward network, the final predicted result can be obtained from a linear layer.

3.2. Analysis of Frequency Compensation Block

Discrete Fourier Transform (DFT):The core idea of the Fourier Transform is to decompose a signal in the time domain into a linear combination of a series of sine and cosine functions. Each sine and cosine function represents a specific frequency component of the signal. Thus, the Fourier transform can help us extract the frequency characteristics from time series data. For discrete signals, the Discrete Fourier Transform is used, and its formula is:

$$X_k={\sum }_{n=0}^{N-1} x_n{e}^{-j2\pi {\displaystyle \frac{k}{N}}n},$$

(9)

where $X_k$ is the complex value of the k-th frequency in the frequency domain; $x_n$ is the n-th sampling point of the time-domain signal; N is the length of the signal. Relatively the IDFT can be defined as:

$$x_n={\sum }_{n=0}^{N-1} X_k{e}^{j2\pi {\displaystyle \frac{k}{N}}n}.$$

(10)

The $\left(9\right)$ shows that for a signal of length N, the computational complexity of the DFT is $O\left(N^2\right)$. However, the Fast Fourier Transform (FFT) reduces the computational load by utilizing the symmetry and periodicity of the signal, breaking the computation into smaller parts, thus reducing its complexity to $O\left(N\mathrm{l}\mathrm{o}\mathrm{g}N\right)$ and significantly improving computational efficiency.

Representation Learning In The Frequency Domain: When performing Fast Fourier Transform on time series data, how to sample is an issue that must be addressed. Retaining all frequency components may inevitably be affected by noise, while preserving only a portion of the frequencies may risk missing some of the underlying trends in the data. FEDformer[28] demonstrates that real-world multivariate time series typically yield low-rank matrices after Fourier transform. This low-rank property implies that representing the time series by randomly selecting a fixed number of Fourier components is reasonable. Consequently, we adopt random sampling as our sampling method and set number of modes as M. After random sampling, the selected set of frequency indices is defined as $\mathcal{I}=\{i_1,i_2,\dots ,i_m\}.$Next, we define two weight tensors, ${\mathcal{W}}^{\left(1\right)}\in {\mathbb{C}}^{F\times N\times N\times M}$ and ${\mathcal{W}}^{\left(2\right)}\in {\mathbb{C}}^{F\times N\times N\times M}$ where $F$ represents the number of features, $N$ represents the number of patches, and $M$ represents the number of frequency components selected from the frequency domain after random sampling. These parameters represent the learnable weights of the network and are initialized with random values. The weights are used to perform weighted transformations on the input signal in the frequency domain. The input patch tensor is $X\in {\mathbb{R}}^{B\times V\times PL\times N}$ where B is the batch size, V is the number of features and PL is the length of each patch. We apply a Fast Fourier Transform (FFT) to the input tensor X along the PL dimension. A tensor $Y_{ft}\in {\mathbb{C}}^{B\times V\times PL\times N//2+1}$ is defined to store the frequency domain data after the Fourier transform. Finally,we use the Inverse Fourier transform (IFFT) to convert the processed frequency domain data back to the time domain. This process can be simply formulated as follows:

$$X_{\mathrm{f}\mathrm{t}}=\mathrm{F}\mathrm{F}\mathrm{T}\left(X\right),$$

(11)

$${\mathcal{W}}_i={\mathcal{W}}_i^{\left(1\right)}+j\cdot {\mathcal{W}}_i^{\left(2\right)},i\in \mathcal{I},$$

(12)

$$Y_{\mathrm{f}\mathrm{t}}={\sum }_i \left(X_{\mathrm{f}\mathrm{t}}\cdot {\mathcal{W}}_i\right),$$

(13)

$$X_{\mathrm{o}\mathrm{u}\mathrm{t}}=IFFT\left(Y_{\mathrm{f}\mathrm{t}}\right),$$

(14)

where $X$ is the input tensor, $X_{\mathrm{f}\mathrm{t}}$ is the result obtained by applying the Fourier transform to $X$, ${\mathcal{W}}_i$ represents the learned complex weight for frequency i. $Y_{\mathrm{f}\mathrm{t}}$ is the frequency domain representation after the weighting operation, and thefinal output is $X_{\mathrm{o}\mathrm{u}\mathrm{t}}\in {\mathbb{R}}^{B\times V\times PL\times N}$.

4. Results

To verify the effectiveness and generality of FCP-Former, we conducted a comprehensive empirical study on eight real-world time-series long-term forecasting datasets, which are widely used in practical applications. To ensure a fair comparison with baseline methods that typically use shorter look-back windows, we set the input length of FCP-Former to 96 as those baselines. This configuration deliberately does not leverage the potential advantage of longer look-back windows afforded by the patching mechanism, focusing instead on the intrinsic capability of the proposed frequency compensation block. The results demonstrate that even under this constrained setting, FCP-Former achieves competitive performance in terms of MSE and MAE compared to existing state-of-the-art methods. Furthermore, we explore the performance of FCP-Former when utilizing longer look-back windows (336 and 512 input time steps), where it demonstrates superior predictive capabilities.

4.1 Experimental Setup

4.1.1. Datasets

We use eight real-world datasets widely used in time series forecasting research. We describe the datasets in detail as follows:

ETT (Electricity Transformer Temperature): consists of two years of data from two different electricity transformers. ETTh1 and ETTh2 are recorded every hour, and ETTm1 and ETTm2 are recorded every 15 minutes.

Traffic: contains data on hourly occupancy rates from 862 sensors of San Francisco Bay area freeways from January 2015 to December 2016.

Weather: provides 21 meteorological factors recorded every 10 minutes at the Weather Station of the Max Planck Biogeochemistry Institute in 2020.

Electricity: records the hourly electricity consumption of 321 customers.

ILI: describes the number of patients and influenza-like illness ratio at weekly intervals, sourced from the US Centers for Disease Control and Prevention between 2002 and 2021.

The statistics of those datasets are summarized in Table 1.

4.1.2. Baselines and Experimental Settings

We choose the SOTA transformer-based model as the baseline, including PatchTST[24], iTransformer[31], TimeXer[32], FEDformer[25], Crossformer[33] and Autoformer[27]. All of the models follow the same experimental setup with prediction length $T\in \left\{\mathrm{24,36,48,60}\right\}$ for ILI dataset and $T\in \{\mathrm{96,192,33}\dot{6},720\}$ for other datasets. To verify the effectiveness and generality of proposed method, we set the input length of the proposed model to 96. This input length is typically used by point-wise methods. The statistics of those baselines are summarized in Table 2.

4.1.3. Metrics

we choose the mean square error(MSE) and mean absolute error(MAE) as evaluation metrics, which can be defined as:

$$MSE={\displaystyle \frac{1}{N}}{\sum }_{t=1}^N (Y_{t_0+t}-{\widehat{Y}}_{t_0+t}{)}^2,$$

(15)

$$MAE={\displaystyle \frac{1}{N}}{\sum }_{t=1}^N \left|Y_{t_0+t}-{\widehat{Y}}_{t_0+t}\right|,$$

(16)

where N is the prediction length, $Y_t$ is the ground truth at timestamp $t$ within the forecast horizon and ${\widehat{Y}}_t$ is the predicted value at timestamp $t$. A lower MSE or MAE indicates better forecasting performance.

4.1.4. Implementation Details

Our model was implemented with PyTorch2.4.0 and trained on an NVIDIA GeForce RTX4090 GPU. We used the Adam optimizer and set the learning rate to 1e-4 to train our model. For small datasets, such as the ETT dataset, we set the batch size to 128. For larger datasets like traffic, due to memory resource limitations, we adjust the batch size between 8 and 32. For all datasets, we set the maximum number of training epochs to 50. To preventoverfitting and reduce training time, we set the dropout rate to 0.05 and use an early stopping mechanism with a patience of 3 to halt training when the validation loss showed no significant decrease. The patch length, denoted as P, was set to 16. The hyperparameter frequency modes, denoted as M, were set to 16.

4.2. Experimental Results

For multivariate forecasting, FCP-Former outperforms other methods on all eight benchmark datasets, as shown in Table 3. The experimental results indicate that FCP-Former significantly outperforms other baseline methods in prediction performance for multivariate long-term time series forecasting tasks. FCP-Former achieves a total of 48 optimal values and 17 suboptimal values, especially on the ETT and electricity datasets. Although FCP-Former does not achieve the best performance on all datasets, it consistently attains near-optimal results. This demonstrates. that FCP-Former exhibits significant advantages in long-term forecasting. It is worth noting that, compared to baseline methods, FCP-Former often exhibits a smaller MAE when the MSE values are similar. This indicates smaller average deviations between predictions and ground truth, reflecting higher overall prediction accuracy. In practical application scenarios, such as stock price forecasting, supply chain management, and healthcare, where a lower MAE is more critical, our model demonstrates a distinct advantage.

4.3. Model Analysis

We will analyze FCP-Former through ablation studies, hyperparameter sensitivity experiments and experiments with different input lengths.

4.3.1. Ablation Studies

In this section, we conduct an ablation study on the model to demonstrate the effectiveness of our frequency compensation block. The following component is ablated:

• w/o FCB: Removing the frequency compensation block before encoding.

We compare the performance of the FCP-Former ablation version and the results of the full FCP-Former model in Table 4. From the results of the ablation experiment, it is evident that the application of the frequency compensation block leads to improved prediction performance.

4.3.2. Hyperparameter Sensitivity Experiments

In frequency compensation block, we employed a crucial hyperparameter: the number of modes in the frequency domain M. This hyperparameter determines how many frequency components are selected from the frequency domain for the model to learn from. Its value directly impacts both the model's frequency domain representation capability and computational complexity. Theoretically, a larger number of modes implies more frequency patterns are used, resulting in higher frequency domain resolution and finer data variations being captured, but at the cost of increased computational load and a higher risk of overfitting. On the other hand, a smaller number of modes compresses the frequency domain information, with the model focusing only on the main low-frequency components. This makes the model lighter and faster, but may lead to the loss of high-frequency information, decreasing representational capacity while potentially improving generalization performance. In our experiments, we evaluate the number of modes in the frequency domain M from the set {2,4,6,8,10,12,14,16,18}. The results are shown in Figure 4. This figure corroborates the aforementioned theoretical analysis. When the value of M is low, the model learns fewer frequency patterns, resulting in relatively lower prediction accuracy. As M increases, the MSE gradually decreases and plateaus. When M reaches 16, the model achieves its optimal performance for this hyperparameter on both the ETTh2 and Electricity datasets. However, as M continues to increase, the model's prediction performance deteriorates due to overfitting, leading to a rise in MSE. This trend of performance deterioration due to overfitting is more pronounced on the Electricity dataset. Considering both computational costs and prediction performance, we recommend setting the value of M to 16.

Figure 4. The MSE results with different number of selected modes in the ETTh1(upper row) and Electricity(lower row) datasets.

Figure 4. The MSE results with different number of selected modes in the ETTh1(upper row) and Electricity(lower row) datasets.

4.3.3. Experiments with Different Input Lengths

In time series forecasting tasks, the input length determines the amount of historical information available to the model. A longer look-back window allows the model to capture a broader range of past observations, thereby expanding its perceptual scope. For a model with strong long-term time dependency modeling corresponding to models referred to as FCP-Former-336 and FCP-Former-512. Since a longer look-back window inevitably leads to increased memory overhead, we dynamically adjusted the batch size to balance memory consumption. Due to the limited size of the ILI dataset, with only 966 data points, increasing the input length leads to a reduction in the training set size. For FCP-Former-512, using the dataset split as shown in Table I, the training set consists of only 8 data points, making training impossible. Similarly, for FCP-Former-336, the training set contains only 184 data points, which is insufficient for adequate model training. Therefore, we did not conduct experiments on the ILI dataset. For the remaining datasets, the comparative results of FCP-Former, FCP-Former-336, and FCP-Former-512 are presented in Table 5. Based on the work of Wang et al[36]., it is evident that due to the presence of repeated short-term patterns in the data, and the difficulty of Transformer models in effectively capturing and modeling these short-term patterns, the performance of Transformer-based models often deteriorates as the input length increases. This phenomenon helps explain why, in a few specific cases, the performance of FCP-Former-336 marginally outperformed that of FCP-Former-512. However, overall, the performance of FCP-Former-512 surpasses that of both FCP-Former-336 and FCP-Former, particularly when the prediction length is higher, where its advantages become even more pronounced. The overall superior performance of FCP-Former-512, particularly for longer prediction horizons, suggests that FCP-Former excels in capturing long-term temporal dependencies and in deeply extracting meaningful information from historical data.

4.4. Multivariate Showcases

As shown in Figure 5, we also compare the prediction results of FCP-Former with those of recently established state-of-the-art models (PatchTST, TimeXer and iTransformer) on the test sets of multiple datasets (ETTm1, Weather, and Electricity). Our model demonstrates predictions that are closest to the ground truth values, providing a direct visual manifestation of its superior MAE performance reported in Table 3. Furthermore, due to its enhanced ability to capture the trend of time series data, FCP-Former exhibits a significant advantage in predicting the overall trend, as evidenced by the close alignment between its predicted trends and the actual trends.

5. Conclusions

In this work, we have proposed a frequency compensation block to optimize the patch-wise transformer-based model in long time series prediction tasks. The frequency compensation block enables the model to perform representation learning of time series data in the frequency domain, enriching the information within each patch of the patch-wise model. This allows the model to more effectively capture the periodic and trend components within the data when performing time series forecasting tasks, and uncovers key information hidden in the raw data. Experimental results on several real-world time series datasets demonstrate that FCP-Former achieves state-of-the-art performance in long-sequence prediction tasks, with predictions that are closer to the true values, better meeting the requirements of practical application scenarios. In future research, we aim to investigate the model's performance on datasets with more chaotic periodic and trend components, to further mitigate the impact of noise on the accuracy of model predictions.