{"type":"script","doc_desc":{"producers":[{"name":"Reproducibility WBG","abbr":"DECDI","affiliation":"World Bank - Development Impact Department","role":"Verification and preparation of metadata"}],"prod_date":"2026-05-08","version":"1"},"project_desc":{"authoring_entity":[{"name":"Leonardo Olivetti","affiliation":"Department of Earth Sciences, Uppsala University; Swedish Centre for Impacts of Climate Extremes (climes); Centre of Natural Hazards and Disaster Science (CNDS)","email":"leonardo.olivetti@geo.uu.se"},{"name":"Gabriele Messori","affiliation":"Department of Earth Sciences, Uppsala University; Swedish Centre for Impacts of Climate Extremes (climes); Department of Meteorology, Stockholm University","email":"gabriele.messori@geo.uu.se"},{"name":"Paolo Avner","affiliation":"World Bank","email":"pavner@worldbank.org"},{"name":"St\u00e9phane Hallegatte","affiliation":"World Bank","email":"shallegatte@worldbank.org"}],"title_statement":{"title":"Reproducibility package for Assessing The Real-World Economic Value Of Weather Forecasts Under Compounding Extremes: A Decision-Specific Framework","idno":"RR_EUR_2026_642"},"data_statement":"All data sources are publicly available and included in the reproducibility package.","software":[{"name":"R","version":"4.4.3 "}],"scripts":[{"title":"Reproducibility package for Assessing The Real-World Economic Value Of Weather Forecasts Under Compounding Extremes: A Decision-Specific Framework","date":"2026-05","notes":"Computational reproducibility verified by Development Impact (DECDI) Analytics team, World Bank. The Reproducibility Package is forthcoming on World Bank Development Data Hub. ","instructions":"See README in reproducibility package.","file_name":"RR_EUR_2026_642","zip_package":"RR_EUR_2026_642.zip","dependencies":"R dependencies are listed in the file renv.lock."}],"repository_uri":[{"name":"Reproducible Research Repository (World Bank)","uri":"https:\/\/reproducibility.worldbank.org"}],"production_date":"2026-05-08","abstract":"Assessing the real-world economic value of weather forecasts remains challenging, particularly in the context of high-impact extreme events. Although meteorological skill has improved substantially in recent years\u2014driven by steady advances in physics-based models and impressive breakthroughs in AI-based forecasting\u2014operational evaluations still focus primarily on standard skill metrics, with limited consideration of how improvements in meteorological skill translate into economic value. In this study, a flexible framework is proposed to assess the economic value of weather forecasts, with penalty functions that explicitly account for compounding losses as well as declining user trust in case of repeated false alarms. In addition, the framework allows for varying cost\u2013loss ratios to represent heterogeneous prevention costs and vulnerability structures. The framework is applied to cities exposed to weather-related natural hazards, comparing the relative economic value of leading physics-based and data-driven forecasting systems from the European Centre for Medium-Range Weather Forecasts (ECMWF). The value of forecasts is highly sensitive to assumptions about compounding losses, penalty structures, and prevention costs\u2014often substantially altering conclusions drawn from meteorological skill alone. For instance, in some cities in Southern Europe, the higher sensitivity of the physics-based model IFS HRES makes it better suited when protection costs are small relative to potential losses, while the higher specificity of the data-driven AIFS makes it better when protection costs are higher. These findings underscore the importance of evaluating economic value under realistic risk scenarios to ensure that improvements in predictive accuracy translate into meaningful societal and economic benefits.\n","geographic_units":[{"name":"Europe","code":"EUR"}],"output":[{"type":"Working Paper","description":"Policy Research Working Papers (PRWP)","title":"Assessing The Real-World Economic Value Of Weather Forecasts Under Compounding Extremes: A Decision-Specific Framework"}],"language":[{"name":"English","code":"EN"}],"disclaimer":"The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development\/The World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.","license":[{"name":"MIT License","uri":"https:\/\/opensource.org\/license\/mit"},{"name":"World Bank IGO Rider","uri":"https:\/\/github.com\/worldbank\/metadata-editor\/blob\/main\/WB-IGO-RIDER.md"}],"contacts":[{"name":"Leonardo Olivetti","affiliation":"Uppsala University","email":"leonardo.olivetti@geo.uu.se"},{"name":"Reproducibility WBG","affiliation":"World Bank","email":"reproducibility@worldbank.org"}],"datasets":[{"name":"ERA5 hourly data on single levels from 1940 to present","note":"Data accessed in 2025. ERA5 reanalysis data retrieved through the Copernicus Climate Data Store. Coverage: 2024-02-28 00:00 to 2025-03-15 18:00 UTC. Timestamps used for forecast verification: 00:00 and 12:00 UTC, matched to step-120 forecast valid times. Variables used: 2-meter temperature (2t), 10-meter U wind component (10u), and 10-meter V wind component (10v). File locations: Data\/t2m_era5.nc; Data\/10u_era5.nc; Data\/10v_era5.nc. An example Python API request is included in Code\/Example_CDS_api_request.py.","access_type":"Data is publicly available and included in the reproducibility package.","license":"Creative Commons Attribution 4.0 International (CC BY 4.0)","license_uri":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/","uri":"https:\/\/cds.climate.copernicus.eu\/datasets\/reanalysis-era5-single-levels?tab=download","citation":"Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Hor\u00e1nyi, A., Mu\u00f1oz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Rozum, I., Schepers, D., Simmons, A., Soci, C., Dee, D., Th\u00e9paut, J-N. (2023): ERA5 hourly data on single levels from 1940 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS), DOI: 10.24381\/cds.adbb2d47. Accessed 2025."},{"name":"ECMWF Forecast Data \u2013 AIFS and IFS HRES ","note":"Data accessed in 2025. Both datasets retrieved through the ECMWF MARS server at step 120 (5-day lead time). Access may require an ECMWF account and member state data access permissions. (1) AIFS forecast data: valid time coverage 2024-03-04 12:00 to 2025-03-06 12:00 UTC; initialization time coverage 2024-02-28 12:00 to 2025-03-01 12:00 UTC. File locations: Data\/AIFS_forecast_t2m_step_120.netcdf; Data\/AIFS_forecast_u10_step_120.netcdf;Data\/ AIFS_forecast_v10_step_120.netcdf. (2) IFS HRES forecast data: valid time coverage 2024-03-04 00:00 to 2025-03-06 12:00 UTC; initialization time coverage 2024-02-28 00:00 to 2025-03-01 12:00 UTC. File locations: Data\/HRES_forecast_t2m_step_120.netcdf; Data\/HRES_forecast_u10_step_120.netcdf;Data\/ HRES_forecast_v10_step_120.netcdf. Variables used for both: 2-meter temperature (t2m), 10-meter U wind component (u10), and 10-meter V wind component (v10). An example Python API request is included in Code\/Example_CDS_api_request.py.","access_type":"Data is publicly available and included in the reproducibility package.","license":"Creative Commons Attribution 4.0 International (CC BY 4.0)","license_uri":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/","uri":"https:\/\/apps.ecmwf.int\/mars-catalogue\/","citation":"European Centre for Medium-Range Weather Forecasts (ECMWF). 2025. \"AIFS and IFS HRES Forecast Data\" [dataset]. ECMWF MARS. https:\/\/apps.ecmwf.int\/mars-catalogue\/. Accessed 2025."}],"reproduction_instructions":"To reproduce the findings in this paper, a replicator must:\n    - Open the R project `replication.Rproj` and restore the environment by running renv::restore() and following the prompts.\n    - Run the files in order: `Theoretical.R` and `Case_studies.R`\n","technology_requirements":"Run time: ~75 minutes","technology_environment":"Paper exhibits were reproduced on a computer with the following specifications:\n\u2022 OS: Windows 11 Enterprise\n\u2022 Processor: INTEL(R) XEON(R) PLATINUM 8562Y+ 2.80 GHz (2 processors)\n\u2022 Memory available: 128.0 GB"},"datacite":{"creators":[{"givenName":"Leonardo","familyName":"Olivetti","nameType":"Personal","affiliation":[{"name":"Uppsala University"},{"name":"Swedish Centre for Impacts of Climate Extremes (climes) "},{"name":"Centre of Natural Hazards and Disaster Science (CNDS)"}]},{"givenName":"Gabriele","familyName":"Messori","nameType":"Personal","affiliation":[{"name":"Uppsala University"},{"name":"Swedish Centre for Impacts of Climate Extremes (climes)"},{"name":"Stockholm University"}]},{"givenName":"Paolo","familyName":"Avner","nameType":"Personal","affiliation":[{"name":"World Bank"}]},{"givenName":"St\u00e9phane","familyName":"Hallegatte","nameType":"Personal","affiliation":[{"name":"World Bank"}]}],"titles":[{"lang":"en","title":"Reproducibility package for Assessing The Real-World Economic Value Of Weather Forecasts Under Compounding Extremes: A Decision-Specific Framework"},{"title":"RR_EUR_2026_642","titleType":"Other"}],"publisher":"World Bank","publicationYear":"2026","types":{"resourceType":"Reproducibility package","resourceTypeGeneral":"Other"},"url":"https:\/\/reproducibility.worldbank.org\/index.php\/catalog\/study\/RR_EUR_2026_642","language":"en"},"tags":[{"tag":"DOI"},{"tag":"Open Code"},{"tag":"Open Data"}],"schematype":"script"}