MeteoFormer: The Accuracy–Faithfulness Trade-off of Built-in Attention in Meteorological Forecasting

Built-in attention is often presented as a free explanation of what a forecasting transformer relies on, but the claim is rarely tested against the model's behaviour. We test it on MeteoFormer, a patch-based transformer with single-head variable and temporal attention that forecasts temperature, relative humidity, pressure and wind speed 24 h ahead from 96 h of hourly input on the Jena climate and Beijing PRSA datasets. Against ten baselines trained under one protocol with five seeds, MeteoFormer has the lowest mean absolute error on Jena (3.025 against 3.107 for the runner-up Crossformer, Diebold-Mariano p < 0.05 against every baseline after Holm correction) and the most accurate temperature forecast on both datasets, while on Beijing it is level with Crossformer in absolute error (4.151 against 4.138, p = 0.72) and Crossformer has the lower squared error, a difference that we trace to the humidity channel. The explanation claim fails. On equal budgets across all eleven models, the fidelity of the built-in variable attention does not differ from zero (0.012 on Jena, 0.046 on Beijing) and is far below post-hoc SHAP on the same model, the temporal attention is almost uniform although occlusion shows that the forecast depends mainly on the last 16 h, and the configuration selected on validation for accuracy has the least faithful attention (0.012 against 0.181 with instance normalization on Jena, p = 0.001).