Reproducibility package for A Data-Driven Approach for Early Detection of Food Insecurity in Yemen's Humanitarian Crisis
Reference ID
RR_YEM_2024_139
Author(s)
Steve Penson, Mathijs Lomme , Bo Pieter Johannes Andree, Zacharey Austin Carmichael , Alemu Manni, Sudeep Shrestha
Collections
Metadata
Created on
Jun 27, 2024
Last modified
Jun 27, 2024
Overview
Abstract
The Republic of Yemen is enduring the world's most severe protracted humanitarian crisis, compounded by conflict, economic collapse, and natural disasters. Current food insecurity assessments rely on expert evaluation of evidence with limited temporal frequency and foresight. This paper introduces a data-driven methodology for the early detection and diagnosis of food security emergencies. The approach optimizes for simplicity and transparency, and pairs quantitative indicators with data-driven optimal thresholds to generate early warnings of impending food security emergencies. Historical validation demonstrates that warnings can be reliably issued before sharp deterioration in food security occurs, using only a few critical indicators that capture inflation, conflict, and agricultural productivity shocks. These indicators signal deterioration most accurately at five months of lead time. The paper concludes that simple data-driven approaches show a strong capability to generate reliable food security warnings in Yemen, highlighting their potential to complement existing assessments and enhance lead time for effective intervention.
Reproducibility Package
Scripts
Title
Reproducibility package (code and partial data) for A Data-Driven Approach for Early Detection of Food Insecurity in Yemen's Humanitarian Crisis
Dependencies
Python: 61 dependencies are used. They are listed with their corresponding version in the file "requirements.txt" of the reproducibility package.
Instructions
See README in reproducibility package.
Notes
Computational reproducibility verified by Development Impact (DIME) Analytics team, World Bank.
Source code repository
| Repository name | URI |
|---|---|
| Reproducible Research Repository | https://reproducibility.worldbank.org |
Software
Name
Python
Version
3.10
Name
Power BI
Version
2.103.881.0 64-bit (March 2022)
Reproducibility
Technology environment
Paper exhibits were reproduced on a computer with the following specifications:
• OS: Windows 11 Enterprise
• Processor: 11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz 1.50 GHz
• Memory available: 15.7 GB
• Software version:
– Python 3.10
– Power BI 2.103.881.0 64-bit (March 2022
Technology requirements
Runtime: 3 hours
Reproduction instructions
Users need to download data from the dataset "Armed Conflict Location & Event Data Project (ACLED)", the Yemen Baseline Assessment and Rapid Displacement Indicators of the IOM, gain access to the dataset "High Resolution Population Density Maps from Facebook for Yemen", and recreate the input file "yemen_adm2_population.csv" to be able to run the code. Instructions for all of these are included in the README of the reproducibility package.
Data
Datasets
Name
FEWS NET Food Security data (2024)
Note
Source: FEWS NET. Filename: "data/preparing master dataset/FEWS February 2024 Update TrueBoundaries_04-19-24.csv". Data was accessed in April 2024.
Access policy
Data is public and included in the reproducibility package
License
Creative Commons Attribution 4.0
Data URL
https://datacatalog.worldbank.org/search/dataset/0064614
Name
Yemen: Acute Food Insecurity Country Data
Note
Source: The IPC Acute Food Insecurity (IPC AFI) classification. Filename: "data/preparing master dataset/ipc_district_2018-2023.csv". Data was accessed in July 2023. The file used for the reproducibility package was manually collapsed by district and groups of months (in the column "Analysis Period") from the file "ipc_yem_area_wide.csv", which is available for download in the data URL.
Access policy
Data is public and included in the reproducibility package
License
Creative Commons Attribution 4.0 International license
License URL
https://creativecommons.org/licenses/by/4.0/deed.en
Data URL
https://data.humdata.org/dataset/b70c2734-2339-4a4d-a69d-fa2bd3225156
Name
FAO Yemen Agricultural Calendar (2023)
Note
Source: Food and Agriculture Organization (FAO) of the United Nations. Filename: "data/preparing master dataset/AgricultualCalendar_Yemen_V2.xlsx". Data was received from Alemu Manni from FAO (Alemu.Manni@fao.org) in July 2023.
Access policy
Data is included in the reproducibility package
Name
Rainfall and SPI data
Note
Source: Food and Agriculture Organization (FAO) of the United Nations. Filename: "data/preparing master dataset/dump_ye_rainfall_and_spi_data.csv". Data was received from Alemu Manni from FAO (Alemu.Manni@fao.org) in July 2023.
Access policy
Data is included in the reproducibility package
Name
Armed Conflict Location & Event Data Project (ACLED)
Note
Source: ACLED. Filename: "data/preparing master dataset/original_acled_conflict.csv". The filename used contains conflict events for Yemen during January 2015 until July 2023. Data was accessed in July 2023.
Access policy
Raw ACLED data cannot be redistributed. Replicators must review the terms of use in the license URL and download the data in the data URL. The dataset is updated weekly and it's possible that data points will not be exactly the same as used by the authors.
License URL
https://acleddata.com/terms-of-use/
Data URL
https://acleddata.com
Name
Monthly food price estimates by product and market
Note
Variable used: food prices, only for Yemen. Data was received over email from the dataset authors on July 2023 along with the datasets Monthly energy price estimates by product and market and Monthly currency exchange rate estimates by market in a wide format by day and market. The dataset can be accessed through the dataset API URL: https://microdata.worldbank.org/index.php/api/tables/data/fcv/wld_2021_rtfp_v02_m or in the data URL below. Filename: Located at: "data/preparing master dataset/world_bank_market_prices.csv".
Access policy
Data is public and included in the reproducibility package.
License
Open data access
License URL
https://microdata.worldbank.org/index.php/terms-of-use
Data URL
https://microdata.worldbank.org/index.php/catalog/4483
Name
Monthly energy price estimates by product and market
Note
Variable used: fuel prices, only for Yemen. Data was received over email from the dataset authors on July 2023 along with the datasets Monthly food price estimates by product and market and Monthly currency exchange rate estimates by market in a wide format by day and market. The dataset can be accessed through the dataset API URL: https://microdata.worldbank.org/index.php/api/tables/data/FCV/WLD_2023_RTEP_v01_M or in the data URL below. Filename: Located at: "data/preparing master dataset/world_bank_market_prices.csv".
Access policy
Data is public and included in the reproducibility package.
License
Open data access
License URL
https://microdata.worldbank.org/index.php/terms-of-use
Data URL
https://microdata.worldbank.org/index.php/catalog/6134
Name
Monthly currency exchange rate estimates by market
Note
Variable used: exchange rate, only for Yemen. Data was received over email from the dataset authors on July 2023 along with the datasets Monthly energy price estimates by product and market and Monthly food price estimates by product and market in a wide format by day and market. The dataset can be accessed through the dataset API URL: https://microdata.worldbank.org/index.php/api/tables/data/FCV/WLD_2023_RTFX_v01_M or in the data URL below. Filename: Located at: "data/preparing master dataset/world_bank_market_prices.csv".
Access policy
Data is public and included in the reproducibility package.
License
Open data access
License URL
https://microdata.worldbank.org/index.php/terms-of-use
Data URL
https://microdata.worldbank.org/index.php/catalog/6160
Name
Telegram Exchange Rate from the Yemen Economic Tracking Initiative
Note
Data was accessed on July 2023 with the API URL call: https://yemen.yeti.acaps.org/api/telegram_exchange_rates/?format=csv+%28non-paginated%29. Data file: "data/preparing master dataset/api_telegram_exchange_rates.csv".
Access policy
Data is public and included in the reproducibility package.
License
Attribution-NonCommercial-NoDerivs 4.0 International
License URL
https://creativecommons.org/licenses/by-nc-nd/4.0/
Data URL
https://yemen.yeti.acaps.org/
Name
Yemen Normalized difference vegetation index (NDVI) at Subnational level
Note
Source: National Aeronautics and Space Administration (NASA) & WFP. Filename: "data/preparing master dataset/yem-ndvi-adm2-full.csv". Data was accessed in July 2023.
Access policy
Data is public and included in the reproducibility package
License
Creative Commons Attribution 4.0 International license
License URL
https://creativecommons.org/licenses/by/4.0/deed.en
Data URL
https://data.humdata.org/dataset/7907439a-0138-4b8e-a4cf-4f0233e0d694/resource/5a30f36a-aa4c-4261-a24b-22bda7639e4c/
Name
Yemen Rainfall Indicators at Subnational Level
Note
Source: WFP. Filename: "data/preparing master dataset/yem-rainfall-adm2-full.csv". Data was accessed in July 2023.
Access policy
Data is public and included in the reproducibility package
License
Creative Commons Attribution 4.0 International license
License URL
https://creativecommons.org/licenses/by/4.0/deed.en
Data URL
https://data.humdata.org/dataset/902bd76c-3f3c-42ad-8547-f9c31deb7937/resource/8ebd5130-715e-4bc9-920c-773d461b0754
Name
Yemen - Baseline Assessment Round 37 (IOM)
Note
Data file: "iom/DTM_YEMEN_Round_37_November_2018.xlsx". Data was accessed in July 2023. Data can be downloaded directly in the URL: https://dtm.iom.int/sites/g/files/tmzbdl1461/files/datasets/DTM_YEMEN_Round_37_November_2018.xlsx.
Access policy
Users must download the data directly from the Data URL provided below.
License URL
https://dtm.iom.int/terms-conditions-use-and-privacy-policy-dtm-website-and-information-products
Data URL
https://dtm.iom.int/datasets/yemen-baseline-assessment-round-37
Name
Yemen — Rapid Displacement Tracking Dataset (25 June - 01 July 2023) (IOM)
Note
Data file: "iom/20230704_RDT Dataset- 25 June 2023 - 01 July 2023 External District Level.xlsx". Data was accessed in July 2023. Data can be downloaded directly in the URL: https://dtm.iom.int/sites/g/files/tmzbdl1461/files/datasets/20230704_RDT%20Dataset-%2025%20June%202023%20-%2001%20July%202023%20External%20District%20Level.xlsx
Access policy
Users must download the data directly from the Data URL provided below.
License URL
https://dtm.iom.int/terms-conditions-use-and-privacy-policy-dtm-website-and-information-products
Data URL
https://dtm.iom.int/datasets/yemen-rapid-displacement-tracking-dataset-25-june-01-july-2023
Name
Yemen — Rapid Displacement Tracking Dataset (25 - 31 December 2022) (IOM)
Note
Data file: "iom/20230102_RDT Dataset- 25 December 2022 - 31 December 2022 External District Level.xlsx". Data was accessed in July 2023. Data can be downloaded directly in the URL: https://dtm.iom.int/sites/g/files/tmzbdl1461/files/datasets/20230102_RDT%20Dataset-%2025%20December%202022%20-%2031%20December%202022%20External%20District%20Level.xlsx
Access policy
Users must download the data directly from the Data URL provided below.
License URL
https://dtm.iom.int/terms-conditions-use-and-privacy-policy-dtm-website-and-information-products
Data URL
https://dtm.iom.int/datasets/yemen-rapid-displacement-tracking-dataset-25-31-december-2022
Name
Yemen — Rapid Displacement Tracking Dataset (02 - 08 January 2022) (IOM)
Note
Data file: "iom/20220109_RDT_Weekly Update- 02 January 2022 - 09 January 2022 External District Level.xlsx". Data was accessed in July 2023. Data can be downloaded directly in the URL: https://dtm.iom.int/sites/g/files/tmzbdl1461/files/datasets/20220109_RDT_Weekly%20Update-%2002%20January%202022%20-%2009%20January%202022%20External%20District%20Level.xlsx
Access policy
Users must download the data directly from the Data URL provided below.
License URL
https://dtm.iom.int/terms-conditions-use-and-privacy-policy-dtm-website-and-information-products
Data URL
https://dtm.iom.int/datasets/yemen-rapid-displacement-tracking-dataset-02-08-january-2022
Name
Yemen — Rapid Displacement Tracking Dataset (01 January - 05 December 2020) (IOM)
Note
Data file: "iom/Yemen - Rapid Displacement Tracking Dataset (01 January - 05 Dec 2020).xlsx". Data was accessed in July 2023. Data can be downloaded directly in the URL: https://dtm.iom.int/sites/g/files/tmzbdl1461/files/datasets/Yemen%20-%20Rapid%20Displacement%20Tracking%20Dataset%20%2801%20January%20-%2005%20Dec%202020%29.xlsx.
Access policy
Users must download the data directly from the Data URL provided below.
License URL
https://dtm.iom.int/terms-conditions-use-and-privacy-policy-dtm-website-and-information-products
Data URL
https://dtm.iom.int/datasets/yemen-%E2%80%94-rapid-displacement-tracking-dataset-01-january-05-december-2020
Name
Yemen — Rapid Displacement Tracking Dataset (01 Jan to 30 Nov 2019) (IOM)
Note
Data file: "iom/DTM Yemen_ Rapid Displacement Tracking (RDT) Dataset _01 Jan to 30 Nov 2019.xlsx". Data was accessed in July 2023. Data can be downloaded directly in the URL: https://dtm.iom.int/sites/g/files/tmzbdl1461/files/datasets/DTM%20Yemen_%20Rapid%20Displacement%20Tracking%20%28RDT%29%20Dataset%20_01%20Jan%20to%2030%20Nov%202019.xlsx
Access policy
Users must download the data directly from the Data URL provided below.
License URL
https://dtm.iom.int/terms-conditions-use-and-privacy-policy-dtm-website-and-information-products
Data URL
https://dtm.iom.int/datasets/yemen-%E2%80%94-rapid-displacement-tracking-dataset-01-jan-02-nov-2019
Name
Yemen Agriculture Zone by Level-2 Administrative Area
Note
Data was received from Alemu Manni from FAO (Alemu.Manni@fao.org) in July 2023. Filename: "data/preparing master dataset/lookupagri_zone_type.csv".
Access policy
Data is included in the reproducibility package
Name
Yemen - Subnational Administrative Boundaries
Note
Source: United Nations Office for the Coordination of Humanitarian Affairs (OCHA). Data was accessed in May 2023. Geographic data from this dataset (“yem_adm_govyem_cso_ochayemen_20191002_GPKG.zip”, available for download in the data URL) was manually merged with other datasets of the project to produce the files "lookup/acled_pcodes.csv", "lookup/adm2_correction-Grid view.csv", "lookup/ipc_pcodes.csv", and "lookup/yemen_pcodes.csv". This data was also manually processed in QGIS to produce datasets with neighboring areas in the files "lookup/neighbouring_adm2_1deg.csv" and "lookup/neighbouring_adm2_2deg.csv" and to produce datasets with spatial joins in "lookup/world_bank_fews_pcodes.csv" and "lookup/yemen_adm2_population.csv".
Access policy
Data is public and included in the reproducibility package
License
Creative Commons Attribution 4.0 International license
License URL
https://creativecommons.org/licenses/by/4.0/deed.en
Data URL
https://data.humdata.org/dataset/cod-ab-yem
Name
High Resolution Population Density Maps from Facebook
Note
Source: Data for Good at Meta. Data for Yemen is not available for download in the data URL. The team received the data for Yemen from Alex Pompe from Meta (alexpompe@fb.com) in July 2023. The data was manually processed in QGIS by mapping it to geographic district data from OCHA and collapsing it at the district and date level, producing the file "lookup/yemen_adm2_population.csv”. This file is not included in the reproducibility package. The README contains more information on how to re-create it for replicators with access to the data for Yemen.
Access policy
Data is restricted. Users must contact Meta to gain access to the data.
Data URL
https://dataforgood.facebook.com/dfg/tools/high-resolution-population-density-maps
Name
Yemen: Areas of control
Note
Source: ACAPS. Data was accessed in June 2023. Data was manually collapsed by the district for the faction controlling the area (column "actors") and the monetary system used (column "irg_or_dfa") in June 2023 to produce the file "lookups/yemen_areas_of_control_adm2.csv", which is included in the reproducibility package.
Access policy
Data is public and included in the reproducibility package
License
Creative Commons Attribution 4.0 International license
License URL
https://creativecommons.org/licenses/by/4.0/deed.en
Data URL
https://data.humdata.org/m/dataset/yemen-areas-of-control?
Data statement
Most data used is public and included in the reproducibility package. Seven datasets are public but cannot be redistributed in the reproducibility package. One dataset is restricted and requires contacting the data owners for access. The README contains information on accessing all datasets.
Description
Output
Type
Working paper
Title
A Data-Driven Approach for Early Detection of Food Insecurity in Yemen's Humanitarian Crisis
Description
Policy Research Working Paper (PRWP)
Authors
| Author | Affiliation | |
|---|---|---|
| Steve Penson | World Bank | spenson@worldbank.org |
| Mathijs Lomme | World Bank, ACAPS | mathijs.lomme@acaps.org |
| Bo Pieter Johannes Andree | World Bank | bandree@worldbank.org |
| Zacharey Austin Carmichael | World Bank | zcarmichael@worldbank.org |
| Alemu Manni | World Bank | alemu.Manni@fao.org |
| Sudeep Shrestha | ACAPS | sds@acaps.org |
Date of production
2024-06
Scope and coverage
Geographic locations
| Location | Code |
|---|---|
| Yemen | YEM |
Keywords
Agriculture
Food Security
Crisis
Early Warning Systems
Food Price Analysis
Vulnerability
Economic Monitoring
Topics
Disclaimer
Disclaimer
The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development/The World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.
Access and rights
License
| Name | URI |
|---|---|
| Modified BSD3 | https://opensource.org/license/bsd-3-clause/ |
Contacts
Contacts
| Name | Affiliation | |
|---|---|---|
| Steve Penson | World Bank | spenson@worldbank.org |
| Reproducibility WBG | World Bank | reproducibility@worldbank.org |
Information on metadata
Producers
| Name | Abbreviation | Affiliation | Role |
|---|---|---|---|
| Reproducibility WBG | DIME | World Bank - Development Impact Department | Verification and preparation of metadata |
Date of Production
2024-06-24
Document version
1