Finite-Sample Precision Limits for Expected Shortfall Forecast Comparisons

1. Introduction

Expected Shortfall (ES) at tail level $\alpha$ is a coherent risk measure widely used in financial regulation, portfolio management, and quantitative risk modelling. Because ES averages losses beyond the $\alpha$-quantile, its estimation draws on at most $n\alpha$ effective tail observations from a calibration sample of size n. At tail levels common in practice—$\alpha \in \{1\%,2.5\%,5\%\}$—this effective count is small even when n is moderately large. The statistical difficulty of ES estimation is therefore governed by $n\alpha$, not by n.

The ${\left(n\alpha \right)}^{-1/2}$ scaling of ES estimation error is well established. Zwingmann and Holzmann [1] derive this rate through a central limit theorem for tail averages, Bartl and Eckstein [2] obtain a concentration lower bound for nonparametric ES estimation, and related kernel estimators [3] obey the same effective-tail-count rate. These results characterise the information limit, but they do not translate it into a practical diagnostic for deciding whether two ES forecasts can be meaningfully distinguished over a finite calibration window.

ES forecast evaluation relies on the joint elicitability of VaR and ES [4]. Two-stage regression frameworks [5] and dynamic semiparametric VaR–ES models [6] provide tools for estimation and evaluation under dependence. Pele et al. [7] show that Fissler–Ziegel-based recalibration can attain the ${\left(n\alpha \right)}^{-1/2}$ rate under geometric mixing. That paper is concerned with achievability of the rate under temporal dependence; the present paper takes the rate as given and converts it into an operational comparison audit comprising the four diagnostics developed in Section 3 and Section 4. ES backtesting from a regulatory perspective is studied by Acerbi and Székely [8] and Nolde and Ziegel [9]. The question addressed here is how much pairwise ES model comparison is statistically supportable once the effective tail count is fixed by the calibration window.

The contribution is operational rather than rate-theoretic. The paper converts the known ES information limit into a precision audit for ES model comparison. The audit has four components:

1.

a plug-in precision benchmark based on the effective tail count, calibrated from tail residuals;

2.

a finite-sample sample-size rule that determines the calibration window needed for a target ES precision tolerance;

3.

a precision-fragile pairwise comparison screen that classifies a forecast pair as precision-fragile when the observed difference in ES recalibration corrections falls below the precision floor implied by the available effective tail count;

4.

a VaR-first diagnostic for attributing excess ES recalibration dispersion to first-stage quantile miscalibration.

The novelty is not the ${\left(n\alpha \right)}^{-1/2}$ rate itself. The novelty is the conversion of this rate into a practical precision audit for ES model comparison. The Le Cam two-point construction in Appendix A (Theorem A1) certifies that no estimator can beat the ${\left(n\alpha \right)}^{-1/2}$ rate over the distribution class, establishing that the plug-in benchmark $\widehat{C}/\sqrt{n\alpha }$ has the correct functional form for a precision floor. The operational constant $\widehat{C}$ is a CLT-motivated plug-in rather than the exact minimax constant $c_L$, but the rate it instantiates is sharp.

The natural application domain for this audit is the Fundamental Review of the Trading Book ([10] FRTB;), which sets ES at $\alpha =2.5\%$ as the capital-determining market-risk measure evaluated over a 250-day calibration window. These parameters imply $n\alpha =6.25$, placing the problem firmly in the finite-sample tail-scarcity regime. FRTB serves as the motivating application throughout this paper, but the audit framework applies to any setting in which ES forecasts are compared over a finite window at a fixed tail level.

The empirical analysis uses a parametric GJR-GARCH-t baseline [11] and three zero-shot time-series foundation models: TimesFM 2.5 [12], Chronos-Small [13], and Moirai 2.0 [14,15]. The forecasters are not ranked as a model-selection exercise. They provide heterogeneous first-stage forecast sequences against which the precision audit can be evaluated. Chronos-Small is retained as a stress case because it exhibits severe first-stage VaR miscalibration, allowing the VaR-first diagnostic to be evaluated under a clear failure mode.

Across 24 global assets and four forecasters evaluated at $\alpha =2.5\%$, between 16% and 29% of pairwise ES rankings fall below the plug-in precision tolerance implied by the effective tail count; the paired block-bootstrap estimate is 22.9%. Excess ES recalibration dispersion is concentrated in cells with poor first-stage VaR calibration. For the median asset, the 250-day calibration window delivers approximately 37 bp of precision, but seven of 24 high-volatility assets require more than 250 days to reach a 50 bp tolerance.

From a mathematical-statistical perspective, the paper studies how an asymptotic information bound for a tail functional can be converted into a finite-sample diagnostic for forecast comparison. The empirical application to market-risk forecasts serves to illustrate the consequences of this effective-sample-size constraint in a realistic dependent time-series setting.

The remainder of the paper is organised as follows. Section 2 develops the theoretical framework, linking the effective tail sample size to ES recalibration through an oracle-equivalence argument. Section 3 defines the plug-in precision benchmark and the operational sample-size rule. Section 4 introduces the precision-fragile comparison screen and the VaR-first diagnostic. Section 5 reports a controlled VaR-miscalibration simulation. Section 6 presents the financial-risk application. Section 7 collects robustness and sensitivity results. Section 8 states the main limitations, and Section 9 concludes.

2. Expected Shortfall as a Tail Functional Under Effective Sample-Size Scarcity

This section establishes the theoretical framework. It defines the effective tail sample size, derives the CLT-based precision scaling, and states the oracle-equivalence argument that transfers the rate to additive ES recalibration.

2.1. Definitions and Effective Tail Sample Size

Let $X_1,\dots ,X_n$ be observations from a distribution P on $\mathbb{R}$ with Lebesgue density f. Fix $\alpha \in (0,1/2)$. The Value-at-Risk at level $\alpha$ is the $\alpha$-quantile ${\mathrm{VaR}}_{\alpha }\left(P\right):=inf\{x:F\left(x\right)\ge \alpha \}$, and the Expected Shortfall is ${\mathrm{ES}}_{\alpha }\left(P\right):={\alpha }^{-1}{\int }_0^{\alpha }{\mathrm{VaR}}_u\left(P\right)du=\mathbb{E}[X\mid X\le {\mathrm{VaR}}_{\alpha }\left(P\right)]$. Both ${\mathrm{VaR}}_{\alpha }$ and ${\mathrm{ES}}_{\alpha }$ are negative for losses throughout; the ${\mathrm{FZ}}_0$ loss in Section 6.1 assumes $e<0$ accordingly.

The ${\left(n\alpha \right)}^{-1/2}$ rate for ES estimation is established by Zwingmann and Holzmann [1] via a CLT for the tail average and by Bartl and Eckstein [2] via a concentration lower bound. The mechanism is the tail identification residual

$$\varphi \left(X\right):=({\mathrm{VaR}}_{\alpha }-X)\mathbb{1}\{X\le {\mathrm{VaR}}_{\alpha }\}/\alpha -({\mathrm{ES}}_{\alpha }-{\mathrm{VaR}}_{\alpha }),$$

(1)

which satisfies $\mathbb{E}\left[\varphi \right]=0$ and $Var\left(\varphi \right)={\sigma }_{\mathrm{tail}}^2/\alpha$, where ${\sigma }_{\mathrm{tail}}^2:=\mathbb{E}[{({\mathrm{VaR}}_{\alpha }-X)}^2\mid X\le {\mathrm{VaR}}_{\alpha }]$. Only an $\alpha$-fraction of observations falls in the tail, and each carries $O(1/\alpha )$ weight in the ES average; the CLT for the sample average of $\varphi \left(X_1\right),\dots ,\varphi \left(X_n\right)$ gives standard error ${\sigma }_{\mathrm{tail}}/\sqrt{n\alpha }$. The effective sample size for ES estimation is therefore $n\alpha$, not n.

2.2. Oracle Equivalence for Additive Recalibration

In the two-stage recalibration framework of Dimitriadis and Bayer [5], a first-stage model produces base VaR and ES forecasts, and the second stage estimates an additive ES correction $r^{*}:={\mathrm{ES}}_{\alpha }\left(P\right)-\overline{E}$ by minimising a Fissler–Ziegel loss [4].

Remark 1 

(Oracle equivalence). Fix a first-stage ES forecast $\overline{E}$ that does not depend on the second-stage calibration sample $X_1,\dots ,X_n$. Estimating $r^{*}\left(P\right):={\mathrm{ES}}_{\alpha }\left(P\right)-\overline{E}$ is equivalent in $L^1$ risk to estimating ${\mathrm{ES}}_{\alpha }\left(P\right)$: since $\overline{E}$ is a fixed constant, ${\mathbb{E}}_P|{\widehat{r}}_n-r^{*}\left(P\right)|={\mathbb{E}}_P|{\widehat{T}}_n-{\mathrm{ES}}_{\alpha }\left(P\right)|$ for ${\widehat{T}}_n:={\widehat{r}}_n+\overline{E}$. Any minimax lower bound for ES estimation therefore transfers directly to $r^{*}\left(P\right)$.

The conceptual chain of the paper has four links: (i) the known ${\left(n\alpha \right)}^{-1/2}$ rate for ES estimation; (ii) oracle equivalence (Remark 1), by which additive ES recalibration inherits this rate under a fixed first-stage forecast; (iii) the plug-in precision benchmark $\widehat{C}/\sqrt{n\alpha }$, which instantiates the rate empirically; and (iv) the operational diagnostics built on this benchmark (Section 6 and Section 7). The novelty lies in links (iii)–(iv), not in the rate itself.

2.3. Distinction Between $c_L$ and $\widehat{C}$

An alternative Le Cam two-point construction yielding a closed-form constant $c_L$ is provided in Appendix A for completeness. The plug-in precision benchmark $\widehat{C}/\sqrt{n\alpha }$ used in the empirics is calibrated from the CLT, not from $c_L$. The distinction matters: $c_L$ is the exact minimax constant from a worst-case two-point argument, whereas $\widehat{C}$ is a CLT-motivated plug-in scale. Throughout, “information limit” and “precision floor” refer to the ${\left(n\alpha \right)}^{-1/2}$ rate; “plug-in benchmark” refers to $\widehat{C}/\sqrt{n\alpha }$.

The fixed-forecast condition in Remark 1 is approximated in the empirical design: the 1000-day GJR-GARCH training window ends before the 250-day calibration window, and the foundation-model weights are not updated on the calibration sample (zero-shot inference). The benchmark is therefore an oracle-style precision diagnostic, not a formal finite-sample guarantee for the full rolling procedure. The operational benchmark $\widehat{C}/\sqrt{n\alpha }$ inherits the rate from oracle equivalence; it does not inherit the exact Le Cam constant $c_L$. When the first stage is mis-specified, the first-stage bias ${\eta }_t$ does not vanish and the oracle equivalence breaks down—precisely the VaR-miscalibration channel that the VaR-first diagnostic detects (Section 4.2).

3. Precision Benchmark and Sample-Size Rule

3.1. Plug-In Precision Benchmark

For each asset, forecaster, and tail level, the plug-in constant is defined as

$$\widehat{C}={\widehat{\sigma }}_{\mathrm{tail}},$$

(2)

the sample standard deviation of the tail residuals

$$\{{\widehat{V}}_t-X_t:X_t\le {\widehat{V}}_t\}.$$

(3)

The corresponding precision benchmark is

$$B_{i,m,\alpha }={\displaystyle \frac{{\widehat{C}}_{i,m,\alpha }}{\sqrt{n\alpha }}}.$$

(4)

This quantity instantiates the effective-tail-count rate in an empirical cell and provides the precision floor against which ES recalibration dispersion is evaluated. The benchmark is an operational diagnostic, not a formal finite-sample confidence interval.

In the main analysis, $\widehat{C}$ is computed once over the full evaluation sample for each cell. This yields an ex-post diagnostic benchmark, not a real-time estimate. In a live implementation, $\widehat{C}$ would instead be estimated on each rolling calibration window. Appendix C reports this rolling-window variant and shows that the qualitative precision-fragile comparison results are preserved, although the absolute benchmark ratios change.

Because $\widehat{C}$ is a sample standard deviation of tail residuals, it carries its own sampling variance, and the precision floor $B=\widehat{C}/\sqrt{n\alpha }$ inherits this uncertainty. The concern is most acute in the rolling-window variant, where ${\widehat{C}}_t$ is re-estimated on each 250-day window from roughly $n\alpha \approx 6.25$ tail points. The main analysis mitigates the problem by computing $\widehat{C}$ once over the full evaluation sample, which aggregates far more than 6.25 tail observations per cell. The rolling-window results in Appendix C (Table A4) confirm that the qualitative conclusions survive when $\widehat{C}$ is instead estimated window-by-window, even though individual benchmark ratios shift—consistent with the additional noise in the floor estimate. A Monte Carlo study in Appendix B (Table A1) confirms that the coefficient of variation of $\widehat{C}$ is 0.76 under Student-$t_5$ at FRTB parameters, declining to 0.43 at $n=1000$. Where the floor itself is uncertain, marginal precision-fragile classifications should be treated as tentative rather than decisive, reinforcing the conservative reading of the screen.

3.2. Empirical Dispersion Measure

The raw rolling-window ES corrections ${\widehat{r}}_n$ contain both local estimation noise and slow-moving volatility-regime shifts. The precision benchmark concerns the former. To isolate within-regime estimation variability, we subtract a 252-day moving average from the correction path before computing its standard deviation. Robustness checks with 126- and 504-day filters in Appendix C show that the forecaster ranking is stable, although absolute ratios vary with the detrending horizon.

For each asset i, forecaster m, and tail level $\alpha$, the empirical benchmark ratio is

$$R_{i,m,\alpha }={\displaystyle \frac{{\widehat{\mathrm{SD}}}^{\det }\left({\widehat{r}}_{i,m,\alpha }\right)}{{\widehat{B}}_{i,m,\alpha }}},$$

(5)

where ${\widehat{B}}_{i,m,\alpha }$ is the plug-in precision benchmark from Equation (4), and ${\widehat{\mathrm{SD}}}^{\det }$ denotes the standard deviation of the detrended correction path. Values $R<1$ indicate that the plug-in scale is conservative for that cell; this does not contradict the Le Cam lower bound, which uses a different constant $c_L$. Values $R>1$ indicate excess dispersion, potentially due to VaR misalignment, non-stationarity, or forecast-extraction noise.

3.3. Finite-Sample Correction

At FRTB parameters ($\alpha =2.5\%$, $n=250$), the effective tail count is $k:=n\alpha =6.25$. Because the number of tail observations is random—binomial$(n,\alpha )$ under correct VaR calibration—the CLT scale requires an inflation factor:

$$f(n,\alpha )=\sqrt{1+{\displaystyle \frac{1-\alpha }{n\alpha }}}.$$

(6)

At FRTB parameters, $f=1.075$, raising the required window by approximately 15%. Monte Carlo calibration (Table 1, 50,000 replications under Student-$t_5$ and GARCH(1,1)-$t_5$ DGPs) shows that the CLT plug-in is accurate to roughly 10–15% at $k=6.25$ under i.i.d. sampling. Under GARCH dynamics, the unconditional ${\sigma }_{\mathrm{tail}}$ can exceed the within-regime value at short windows, producing $\widehat{f}<1$; this motivates the detrending step in Section 3.2. Full details are in Appendix B.

3.4. Operational Sample-Size Rule

The Le Cam constant $c_L$ in Appendix A depends on local density geometry and is not directly estimable. For operational sample-size calculations, we therefore instantiate the ${\left(n\alpha \right)}^{-1/2}$ rate with the CLT-motivated plug-in scale $C={\widehat{\sigma }}_{\mathrm{tail}}$, estimated from tail residuals [see [16], Ch. 6]. Incorporating the random-count correction from Section 3.3, the required window length for tolerance $\epsilon$ solves

$${\displaystyle \frac{f(n,\alpha )C}{\sqrt{n\alpha }}}\le \epsilon ⟺n\ge {\displaystyle \frac{f{(n,\alpha )}^2}{\alpha }}{\left({\displaystyle \frac{C}{\epsilon }}\right)}^2,$$

(7)

where

$$f(n,\alpha )=\sqrt{1+{\displaystyle \frac{1-\alpha }{n\alpha }}}.$$

(8)

Because $f(n,\alpha )$ depends on n, the rule is implicit; in practice, fixed-point iteration converges in a few steps. At $\alpha =2.5\%$ and $n=250$, the correction factor is $f(250,0.025)=1.075$, which raises the required window by approximately 15%. For $k=n\alpha \ge 25$, the correction is below 2% and can usually be ignored.

Table 2 reports the implied window lengths for four tail levels and four tolerances, calibrated from a Student-$t_5$ reference distribution. The table has two immediate implications. First, sub-FRTB tail levels are effectively infeasible on realistic calibration windows: at $\alpha =0.5\%$ and $\epsilon =10\%$, the required window is about 44,700 trading days. Second, at the FRTB level $\alpha =2.5\%$, a 250-day window implies a corrected tolerance of approximately $0.43C$, which is of the same order as many inter-model ES differences.

Asset-specific calibrations are reported in Table A11 in Appendix I and summarised in Section 6. Empirical ratios should be interpreted as distances to the CLT-motivated plug-in benchmark, not as tests of the exact Le Cam constant.

4. Precision-Fragile Pairwise Comparison Screen

4.1. Screen Definition

We call a comparison precision-fragile if its absolute ES-recalibration difference is smaller than the plug-in tolerance. The screen is not a hypothesis test and does not attach a nominal size. It is a conservative diagnostic: it asks whether the observed difference is larger than the precision floor implied by the available effective tail count.

The threshold is forecaster-agnostic, conditional on approximate first-stage calibration: it depends on the per-cell tail dispersion $\widehat{C}$ and the effective tail count $n\alpha$, not on the model class itself. Adding or substituting forecasters changes the set of pairwise comparisons, but not the precision floor used to evaluate each pair. Severely miscalibrated forecasters can still inflate the precision-fragile share through the VaR-miscalibration channel, which the VaR-first diagnostic examines in Section 6.3.

Formally, for a given asset and tail level $\alpha$, the comparison between forecasters 1 and 2 is classified as precision-fragile when

$$\left|{\overline{r}}_n^{\left(1\right)}-{\overline{r}}_n^{\left(2\right)}\right|<{\displaystyle \frac{\sqrt{{\widehat{C}}_1^2+{\widehat{C}}_2^2}}{\sqrt{n\alpha }}},$$

(9)

where ${\overline{r}}_n^{\left(j\right)}$ denotes the mean ES recalibration correction for forecaster j over the relevant evaluation windows. The threshold treats the two correction estimates as independently noisy. Because the same return realisation enters both forecasters’ recalibration losses, pairwise correction estimates are likely positively correlated. Ignoring this covariance inflates the tolerance and therefore classifies marginal cases as precision-fragile; the screen is conservative by design.

Although the algebraic form of the threshold resembles a pooled standard error, its substantive content differs in three respects. First, the scale is anchored to the effective tail count $n\alpha$—6.25 observations at FRTB parameters, not the 250 days that constitute the nominal window—so the tolerance reflects the information actually available in the tail rather than the total calibration length. Second, the precision floor is a fixed comparison standard: it is invariant to the forecaster set, whereas the realised precision-fragile share depends on which models are compared. Third, the screen is offered as a conservative diagnostic, not a hypothesis test with nominal size; it flags comparisons whose observed difference is smaller than the precision floor, without attaching a rejection probability.

4.2. Rate Tests and VaR-First Diagnostic

Three diagnostics assess whether the ${\left(n\alpha \right)}^{-1/2}$ rate governs empirical recalibration dispersion and whether deviations from the benchmark are linked to first-stage quantile failure.

First, a cross-sectional scaling regression across all (asset, forecaster, $\alpha$) cells:

$$log\left({\widehat{\mathrm{SD}}}^{\det }\left({\widehat{r}}_n\right)\right)=a+blog\left(n\alpha \right)+\gamma log\left({\widehat{\sigma }}_{\mathrm{tail}}\right)+u.$$

(10)

The rate prediction is $b=-1/2$, while the CLT plug-in scale implies $\gamma =1$. Standard errors are clustered by asset.

Second, a non-overlapping window-length scaling test at $\alpha =1\%$, where variation in the effective tail count is most visible. The calibration length is varied over

$$n\in \{250,500,750,1000\},$$

(11)

with step size equal to n, so adjacent windows do not overlap. For each (asset, forecaster) pair, we estimate

$$log\left(\widehat{\mathrm{SD}}\left({\widehat{r}}_n\right)\right)=a+blog\left(n\right)+u,$$

(12)

using HC1 robust standard errors. The pooled fixed-effects specification is

$$log\left(\widehat{\mathrm{SD}}\left({\widehat{r}}_{i,j,n}\right)\right)={\alpha }_i+{\beta }_j+blog\left(n\right)+u_{i,j,n},$$

(13)

where ${\alpha }_i$ and ${\beta }_j$ denote asset and forecaster fixed effects.

Third, the VaR-first diagnostic distinguishes intrinsic ES estimation noise from first-stage quantile failure. For each (asset, forecaster) pair, VaR backtesting statistics are computed at $\alpha =1\%$. The main diagnostic uses the Christoffersen conditional-coverage statistic [17]; the Kupiec unconditional-coverage statistic [18] is reported as a robustness check. Excess ES recalibration dispersion should concentrate in cells with poor first-stage VaR calibration.

5. Simulation Evidence

This section isolates the mechanism behind the VaR-first diagnostic. The empirical results in Section 6 show that excess ES recalibration dispersion is concentrated in poorly VaR-calibrated cells. The simulation below asks whether this pattern can arise when the first-stage VaR model fails to track time-varying volatility.

The simulation generates 30 independent paths of length 10,000 from a GARCH(1,1)-$t_5$ process with $\omega ={10}^{-6}$, ${\alpha }_g=0.10$, and ${\beta }_g=0.85$. For each path, the first-stage VaR/ES model uses a convex blend of the true conditional volatility ${\sigma }_t$ and the unconditional volatility $\overline{\sigma }$:

$${\sigma }_{\mathrm{model},t}=(1-\delta ){\sigma }_t+\delta \overline{\sigma },\delta \in \{0,0.70,0.85,1\}.$$

(14)

At $\delta =0$, the model uses the true conditional volatility. At $\delta =1$, it ignores conditional volatility dynamics and uses the unconditional volatility. Additive ES corrections are estimated on rolling 250-day windows, advanced in 21-day steps, at $\alpha =2.5\%$. The ES dispersion ratio R is computed as the raw standard deviation of the ES correction divided by the oracle benchmark

$${\displaystyle \frac{{\sigma }_{\mathrm{tail}}}{\sqrt{n\alpha }}}.$$

(15)

Table 3 shows that R increases from 0.95 under the oracle volatility specification to 1.76 under the unconditional-volatility specification. The hit rate rises at the same time, indicating worsening VaR calibration. The transition through $R=1$ occurs between the $\delta =0.70$ and $\delta =0.85$ designs, where the hit rate is already far above the nominal 2.5%. The simulation therefore supports the interpretation that regime-dependent VaR error can inflate ES recalibration dispersion, consistent with the VaR-first diagnostic.

6. Financial-Risk Application

This section illustrates the precision-audit framework on 24 global financial assets and four forecasters under FRTB calibration parameters. The forecasters are not ranked as a model-selection exercise; they provide heterogeneous first-stage forecast sequences against which the precision audit can be evaluated.

6.1. Data and Forecasting Setup

The empirical analysis uses daily log returns for 24 global assets: equity indices, bonds, commodities, cryptocurrencies, and currencies. The sample period is 2002–2026, with exact start and end dates varying by asset; details are reported in Appendix C. Forecasts are computed at $\alpha \in \{1\%,2.5\%,5\%\}$, with $\alpha =2.5\%$ as the primary ES analysis level. Additive Fissler–Ziegel recalibration is performed on rolling $n=250$-day calibration windows, advanced in monthly steps of 21 trading days.

The parametric benchmark is a GJR-GARCH(1,1) model with Student-t innovations [11], estimated by maximum likelihood on a rolling 1000-day training window, with VaR and ES computed analytically from the fitted conditional distribution. The remaining forecasters are time-series foundation models run zero-shot. TimesFM 2.5 [12] uses nine quantile heads with a Student-t fit by quantile matching. Chronos-Small [13] generates 1000 Monte Carlo forecast samples, from which empirical quantiles and conditional tail means yield VaR and ES directly. Moirai 2.0 [14,15] produces 1000 forecast samples with a Student-t fit to extract VaR and ES. Implementation details of the extraction procedures are reported in Appendix C.

Given base forecasts $({\widehat{V}}_t,{\widehat{E}}_t)$, the second stage estimates an additive correction pair $({\widehat{q}}_n,{\widehat{r}}_n)$ by minimising the Fissler–Ziegel ${\mathrm{FZ}}_0$ loss [4,5]. The corrected forecasts are

$${\tilde{V}}_t={\widehat{V}}_t+{\widehat{q}}_n,{\tilde{E}}_t={\widehat{E}}_t+{\widehat{r}}_n,$$

(16)

where ${\widehat{q}}_n$ and ${\widehat{r}}_n$ are held fixed within each calibration window. The loss function is

$$S(v,e,x)={\displaystyle \frac{\mathbb{1}\{x\le v\}(x-v)}{\alpha e}}+{\displaystyle \frac{v}{e}}+log(-e)-1,e<0.$$

(17)

The optimisation uses Nelder–Mead with three random restarts per window. Convergence failure is below 0.5% across all cells.

6.2. Precision-Fragile ES Comparisons

At the FRTB ES level $\alpha =2.5\%$, 29 of 144 pairwise comparisons are precision-fragile under the plug-in screen in Equation (9), corresponding to a share of 20.1% (Table 4). The share increases as the tail level becomes more extreme, consistent with the ${\left(n\alpha \right)}^{-1/2}$ precision limit.

The 20.1% headline should be interpreted as a baseline diagnostic, not as a universal constant. Table 5 reports the precision-fragile share under alternative noise scales and detrending choices. At $\alpha =2.5\%$, the share ranges from 16.0% to 29.2%, with the paired block bootstrap at 22.9%.

Table 6 summarises the three main choices for estimating the precision scale at $\alpha =2.5\%$. The qualitative conclusion is unchanged: a material fraction of ES comparisons is smaller than the precision floor supported by the available tail data.

The sample-size rule reveals substantial cross-asset heterogeneity. At $\alpha =2.5\%$, the estimated tail-dispersion scale spans more than an order of magnitude: USD/JPY requires roughly 84 trading days for 50 bp precision, whereas BTC requires roughly 4,900 days. For the median asset, the 250-day calibration window delivers approximately 37 bp precision. Seven of 24 assets require more than 250 days to reach a 50 bp tolerance, so the sample-size rule is as much an asset-selection diagnostic as a window-length diagnostic.

6.3. Empirical Diagnostics

Figure 1 compares the detrended standard deviation of the ES correction ${\widehat{r}}_n$ with the plug-in precision benchmark $\widehat{C}/\sqrt{n\alpha }$ across all (asset, forecaster, $\alpha$) cells. The 45-degree line corresponds to $R=1$. Points below the line indicate that the plug-in benchmark is conservative for that cell; points above the line indicate excess dispersion, which is examined through the VaR-first diagnostic below.

Figure A3 in Appendix I disaggregates the same comparison by forecaster. GJR-GARCH-t and Moirai 2.0 mostly lie below the 45-degree line, while Chronos-Small straddles it.

The cross-sectional scaling regression in Equation (10) tests whether empirical ES recalibration dispersion follows the predicted effective-tail-count rate. Table 7 reports the results across 288 cells. The estimated slope is $\widehat{b}=-0.436$ with clustered standard error $0.039$, close to the theoretical value $-1/2$. The tail-dispersion coefficient is $\widehat{\gamma }=0.870$ with standard error $0.036$, below the plug-in prediction $\gamma =1$.

The estimate $\widehat{\gamma }<1$ does not contradict the ${\left(n\alpha \right)}^{-1/2}$ rate, which concerns scaling in the effective tail count. Two mechanisms can reduce the estimated tail-dispersion coefficient: full-sample estimation of ${\widehat{\sigma }}_{\mathrm{tail}}$ may average across volatility regimes, and rolling-window overlap may reduce the measured dispersion of ${\widehat{r}}_n$.

Table 8 reports the benchmark ratio R at $\alpha =2.5\%$. GJR-GARCH-t, TimesFM 2.5, and Moirai 2.0 have median ratios below one, whereas Chronos-Small has median $R=1.13$ and reaches $R=2.69$ for Natural Gas. This pattern is consistent with excess ES recalibration dispersion being linked to first-stage VaR failure rather than to the precision benchmark itself.

The controlled simulation in Section 5 establishes the mechanism: as the first-stage volatility model moves from the oracle specification ($\delta =0$) toward the unconditional-volatility specification ($\delta =1$), VaR calibration degrades and the ES dispersion ratio rises from $R=0.95$ to $R=1.76$ (Table 3). This regime-dependent VaR error channel is independent of any single forecaster and provides the causal logic for the VaR-first diagnostic.

The empirical data corroborate this mechanism. Figure 2 plots the Christoffersen conditional-coverage statistic against R at $\alpha =1\%$. The pooled Spearman correlation is $\rho =0.776$ ($p<0.001$). Excluding Chronos-Small, the correlation remains positive and significant ($\rho =0.513$, $p<0.001$), whereas within Chronos-Small alone it is small and insignificant. All 12 cells with $R>1$ belong to Chronos-Small and exhibit severe VaR miscalibration (Table A16 in Appendix I). The pooled association is strengthened by the separation of Chronos-Small from the remaining forecasters, and the ex-Chronos correlation becomes insignificant at $\alpha =2.5\%$ and $5\%$ (Table 11), consistent with the reduced power of the diagnostic at less extreme tail levels.

Chronos-Small is therefore best interpreted as a stress case for the diagnostic, not as a competitive benchmark. Its excess dispersion is consistent with time-varying VaR misalignment; after separating these severely miscalibrated cells, ES recalibration dispersion is broadly comparable across the remaining forecasters.

Table 9 reports the headline results under forecaster subsets. Excluding Chronos-Small increases the precision-fragile share because the remaining forecasters produce ES corrections that are harder to distinguish. The precision-fragile share is not a model-free empirical constant; it depends on the dispersion of the forecasters included in the comparison set. The forecaster-agnostic object is the precision threshold, not the realised share.

6.4. Implications for Tail-Risk Practice

At $\alpha =2.5\%$ with $n=250$ and a typical $\widehat{C}\approx 1.3\%$, the implied recalibration tolerance is

$${\epsilon }_n={\displaystyle \frac{\widehat{C}}{\sqrt{n\alpha }}}\approx 52\mathrm{bp}.$$

(18)

This tolerance is of the same order as many inter-model ES differences, so it provides a simple way to report the sampling uncertainty attached to ES recalibration and model comparison.

The precision audit can be implemented as a four-step workflow:

1.

report the effective tail count $n\alpha$, not only the window length n;

2.

estimate $\widehat{C}={\widehat{\sigma }}_{\mathrm{tail}}$ from tail residuals;

3.

compute the plug-in precision floor $\widehat{C}/\sqrt{n\alpha }$;

4.

flag ES comparisons whose absolute recalibration difference falls below the corresponding pairwise tolerance as precision-fragile.

A precision-fragile comparison does not imply that the two models are equivalent. It means that the observed ES-recalibration difference is smaller than the precision budget supported by the available tail data. Such comparisons should not be used as decisive evidence for model replacement without additional data, stronger structural assumptions, or complementary diagnostics. Appendix J summarises the benchmark, sample-size rule, and diagnostic screen as a workflow.

7. Robustness and Sensitivity

This section collects robustness checks on the main empirical findings.

Cutoff multiplier sensitivity. Replacing the baseline tolerance in Equation (9) by $\kappa \sqrt{{\widehat{C}}_1^2+{\widehat{C}}_2^2}/\sqrt{n\alpha }$ gives the shares in Table 10. Even at $\kappa =0.5$, 16.0% of comparisons remain precision-fragile.

VaR-first diagnostic across tail levels.Table 11 reports the diagnostic at all three tail levels. The association between the conditional-coverage statistic and R is strongest at $\alpha =1\%$, supporting the choice of this level as the diagnostic anchor. At $\alpha =2.5\%$ and $5\%$, the full-sample correlation remains significant, but the correlation excluding Chronos-Small becomes insignificant.

Table 11. VaR-first diagnostic robustness across tail levels. Entries report Spearman rank correlations between the Christoffersen conditional-coverage statistic and benchmark ratio R at the diagnostic $\alpha$ indicated. *** denotes $p<0.001$.

Diagnostic $\mathit{\alpha }$	n pairs	Spearman $\mathit{\rho }$ (all)	Spearman $\mathit{\rho }$ (excl. Chronos)	Comment
1%	76	$0.776***$	$0.513***$	FRTB VaR gatekeeper
2.5%	87	$0.590***$	$0.192$	Matched to FRTB ES level
5.0%	95	$0.559***$	$0.146$	Scaling validation

Table 11. VaR-first diagnostic robustness across tail levels. Entries report Spearman rank correlations between the Christoffersen conditional-coverage statistic and benchmark ratio R at the diagnostic $\alpha$ indicated. *** denotes $p<0.001$.

Diagnostic $\mathit{\alpha }$	n pairs	Spearman $\mathit{\rho }$ (all)	Spearman $\mathit{\rho }$ (excl. Chronos)	Comment
1%	76	$0.776***$	$0.513***$	FRTB VaR gatekeeper
2.5%	87	$0.590***$	$0.192$	Matched to FRTB ES level
5.0%	95	$0.559***$	$0.146$	Scaling validation

Window-length scaling. A non-overlapping window-length scaling test at $\alpha =1\%$ provides a complementary check; details are reported in Appendix D. Three of four forecasters are consistent with the ${\left(n\alpha \right)}^{-1/2}$ rate. The pooled test rejects $H_0:b=-0.50$ at $p=0.033$, driven by TimesFM 2.5; excluding TimesFM 2.5, all subsamples are consistent with the theoretical slope within their 95% confidence intervals.

HS-250 naive benchmark. As a robustness check, Appendix H reports results for a naive Historical Simulation benchmark (HS-250) that uses the empirical quantile and tail mean from the same 250-day window. HS-250 lies close to the plug-in precision floor ($R=0.96$ at $\alpha =2.5\%$), and all of its comparisons with TimesFM-2.5 and Moirai-2.0 are precision-fragile, reinforcing the interpretation that the binding constraint is often the effective tail count rather than model complexity.

8. Limitations

Six limitations delimit the interpretation.

First, the empirical benchmark uses the CLT-motivated plug-in scale $\widehat{C}$, not the Le Cam constant $c_L$. The latter depends on local density geometry and is not estimated from data. Hence values $R<1$ indicate that the plug-in benchmark is conservative for that cell; they do not contradict the lower-bound argument.

Second, the non-overlapping window-scaling test is data-intensive. At $n=1000$, it requires roughly 5,000 trading days, leaving too few usable assets for Chronos-Small to estimate a meaningful model-specific slope.

Third, the theoretical benchmark is oracle-style: it is conditional on a fixed first-stage forecast sequence. The empirical rolling design approximates this condition because forecasts are predictable from past information, but they are not independent of the full calibration path. A strictly held-out design would isolate the oracle assumption more cleanly.

Fourth, absolute benchmark ratios depend on how low-frequency volatility-regime shifts are removed. The forecaster ranking is stable across the 126-, 252-, and 504-day filters reported in Appendix C, but the levels of R and the precision-fragile share vary with the detrending choice.

Fifth, the pooled window-scaling test rejects $H_0:b=-0.50$ at $p=0.033$, driven by TimesFM 2.5 (Appendix D). The rejection disappears in restricted samples.

Sixth, the VaR-first diagnostic is partly a between-group result. The pooled association between the Christoffersen conditional-coverage statistic and R is strengthened by the separation of Chronos-Small from the remaining forecasters.

These limitations define the scope of the audit. The precision floor is a necessary condition for meaningful ES comparison, not a sufficient condition for model superiority. The proposed quantities should be read as precision diagnostics for ES model comparison under tail-data scarcity.

9. Conclusions

This paper studies how the effective tail sample size constrains pairwise comparison of Expected Shortfall forecasts. The relevant precision budget is $n\alpha$, not the nominal calibration window length n. At the FRTB tail level $\alpha =2.5\%$ and $n=250$, the effective tail count is only $n\alpha =6.25$.

The paper converts the known ${\left(n\alpha \right)}^{-1/2}$ information limit into a precision-audit framework with four components: a plug-in precision benchmark, a finite-sample sample-size rule, a precision-fragile pairwise comparison screen, and a VaR-first diagnostic. The audit is diagnostic, not a replacement for formal model validation.

Across 24 global assets and four forecasters, roughly one in five pairwise ES comparisons is precision-fragile: the observed difference in recalibrated ES corrections is smaller than the precision floor supported by the available tail data. This share ranges from 16% to 29% across estimation variants and remains economically material across alternative noise scales, detrending methods, and bootstrap specifications.

The VaR-first diagnostic shows that excess ES recalibration dispersion is concentrated in cells with poor first-stage VaR calibration. VaR miscalibration is therefore a major channel for inflated ES dispersion, consistent with the simulation evidence in Section 5. A naive Historical Simulation benchmark lies close to the plug-in precision floor, reinforcing the interpretation that at standard tail levels the effective number of tail observations can be more binding than model complexity.

The precision-fragile share is not a universal constant. It depends on the asset universe, forecaster set, tail-dispersion estimate, and detrending convention. What is invariant is the information constraint: ES recalibration precision scales with $n\alpha$, not with the nominal window length alone.

The analysis is an ex-post oracle-style precision audit, not a deployable real-time rule. Future work should develop fully held-out, real-time, dependence-aware versions of the audit that relax the fixed-first-stage assumption. Extensions to multivariate tail-risk settings and to formal decision-theoretic frameworks for the comparison screen are also left for future investigation.