Preprint Article Version 1 This version not peer reviewed

Evaluation of Analysis by Cross-Validation. Part I: Using Verification Metrics

Version 1 : Received: 23 January 2018 / Approved: 23 January 2018 / Online: 23 January 2018 (16:23:25 CET)

How to cite: Menard, R.; Deshaies-Jacques, M. Evaluation of Analysis by Cross-Validation. Part I: Using Verification Metrics. Preprints 2018, 2018010217 (doi: 10.20944/preprints201801.0217.v1). Menard, R.; Deshaies-Jacques, M. Evaluation of Analysis by Cross-Validation. Part I: Using Verification Metrics. Preprints 2018, 2018010217 (doi: 10.20944/preprints201801.0217.v1).

Abstract

We examine how observations can be used to evaluate an air quality analysis by verifying against passive observations (i.e. cross-validation) that are not used to create the analysis and we compare these verifications to those made against the same set of (active) observations that were used to generate the analysis. The results show that both active and passive observations can be used to evaluate of first moment metrics (e.g. bias) but only passive observations are useful to evaluate second moment metrics such as variance of observed-minus-analysis and correlation between observations and analysis. We derive a set of diagnostics based on passive observation–minus-analysis residuals and we show that the true analysis error variance can be estimated, without relying on any statistical optimality assumption. This diagnostic is used to obtain near optimal analyses that are then used to evaluate the analysis error using several different methods. We compare the estimates according to the method of Hollingsworth Lonnberg, Desroziers, a diagnostic we introduce, and the perceived analysis error computed from the analysis scheme, to conclude that as long as the analysis is optimal, all estimates agrees within a certain error margin. The analysis error variance at passive observation sites is also obtained.

Subject Areas

data assimilation; statistical diagnostics of analysis residuals; estimation of analysis error, air quality model diagnostics; Desroziers method; cross-validation

Readers' Comments and Ratings (0)

Leave a public comment
Send a private comment to the author(s)
Rate this article
Views 0
Downloads 0
Comments 0
Metrics 0
Leave a public comment

×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.