Ben Bouallègue, Z.; Richardson, D.S. On the ROC Area of Ensemble Forecasts for Rare Events. Weather and Forecasting 2022, 37, 787–796, doi:10.1175/waf-d-21-0195.1.
Ben Bouallègue, Z.; Richardson, D.S. On the ROC Area of Ensemble Forecasts for Rare Events. Weather and Forecasting 2022, 37, 787–796, doi:10.1175/waf-d-21-0195.1.
Ben Bouallègue, Z.; Richardson, D.S. On the ROC Area of Ensemble Forecasts for Rare Events. Weather and Forecasting 2022, 37, 787–796, doi:10.1175/waf-d-21-0195.1.
Ben Bouallègue, Z.; Richardson, D.S. On the ROC Area of Ensemble Forecasts for Rare Events. Weather and Forecasting 2022, 37, 787–796, doi:10.1175/waf-d-21-0195.1.
Abstract
The relative operating characteristic (ROC) curve is a popular diagnostic tool in forecast verification, with the area under the ROC curve (AUC) used as a verification metric measuring the discrimination ability of a forecast. Along with calibration, discrimination is deemed as a fundamental probabilistic forecast attribute. In particular, in ensemble forecast verification, AUC provides a basis for the comparison of potential predictive skill of competing forecasts. While this approach is straightforward when dealing with forecasts of common events (e.g. probability of precipitation), the AUC interpretation can turn out to be oversimplistic or misleading when focusing on rare events (e.g. precipitation exceeding some warning criterion). How should we interpret AUC of ensemble forecasts when focusing on rare events? How can changes in the way probability forecasts are derived from the ensemble forecast affect AUC results? How can we detect a genuine improvement in terms of predictive skill? Based on verification experiments, a critical eye is cast on the AUC interpretation to answer these questions. As well as the traditional trapezoidal approximation and the well-known bi-normal fitting model, we discuss a new approach which embraces the concept of imprecise probabilities and relies on the subdivision of the lowest ensemble probability category.
Computer Science and Mathematics, Probability and Statistics
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
