Reproducibility package for Assessing The Real-World Economic Value Of Weather Forecasts Under Compounding Extremes: A Decision-Specific Framework

Assessing the real-world economic value of weather forecasts remains challenging, particularly in the context of high-impact extreme events. Although meteorological skill has improved substantially in recent years—driven by steady advances in physics-based models and impressive breakthroughs in AI-based forecasting—operational evaluations still focus primarily on standard skill metrics, with limited consideration of how improvements in meteorological skill translate into economic value. In this study, a flexible framework is proposed to assess the economic value of weather forecasts, with penalty functions that explicitly account for compounding losses as well as declining user trust in case of repeated false alarms. In addition, the framework allows for varying cost–loss ratios to represent heterogeneous prevention costs and vulnerability structures. The framework is applied to cities exposed to weather-related natural hazards, comparing the relative economic value of leading physics-based and data-driven forecasting systems from the European Centre for Medium-Range Weather Forecasts (ECMWF). The value of forecasts is highly sensitive to assumptions about compounding losses, penalty structures, and prevention costs—often substantially altering conclusions drawn from meteorological skill alone. For instance, in some cities in Southern Europe, the higher sensitivity of the physics-based model IFS HRES makes it better suited when protection costs are small relative to potential losses, while the higher specificity of the data-driven AIFS makes it better when protection costs are higher. These findings underscore the importance of evaluating economic value under realistic risk scenarios to ensure that improvements in predictive accuracy translate into meaningful societal and economic benefits.