Home / Repository / PRWP / RR_WLD_2026_639
PRWP

Reproducibility package for Innovation Patterns In The World Bank Group Portfolio

2026
Get Reproducibility Package
Reference ID
RR_WLD_2026_639
DOI
https://doi.org/10.60572/f3ma-2q12
Author(s)
Natalia Agapitova, Anastasia Nedayvoda, Stephen Winkler, Amschel de Rothschild, Ergun Ertekin, Sarah Lenoble
Collections
World Bank Policy Research Working Papers
Metadata
JSON
Created on
May 05, 2026
Last modified
May 06, 2026
Page views
3
  • Overview
  • Reproducibility Package
  • Description
  • Scope and coverage
  • Disclaimer
  • Access and rights
  • Contacts
  • Information on metadata
  • Citation

    • Overview

    Abstract

    This paper employs large-scale text analytics and generative-AI methods to analyze innovation patterns within the World Bank Group (WBG) portfolio. It draws on over 7,500 Independent Evaluation Group (IEG) project evaluations completed between 1998 and 2025. The study finds that, on average, innovative projects are associated with higher performance outcomes. It also identifies several key enabling factors for innovation at the WBG, including: disciplined experimentation, flexible and adaptive project designs, contextualization to local settings, participatory approaches in the design and piloting of new solutions, and visionary and supportive internal leadership. The analysis reveals that innovation within the WBG has evolved over time from sporadic experimentation to an embedded organizational capability. However, significant constraints persist such as bureaucratic fragmentation, capacity gaps, overly complex project designs, lack of continuity in scaling innovations, and insufficient incentives for learning and adaptation.

    Reproducibility Package

    Scripts
    Get Reproducibility Package
    File name
    RR_WLD_2026_639
    Zip package
    RR_WLD_2026_639.zip
    Title
    Reproducibility package for Innovation Patterns In The World Bank Group Portfolio
    Date
    2026-05
    Instructions
    See README in reproducibility package.
    Notes
    Computational reproducibility verified by Development Impact (DECDI) Analytics team, World Bank.
    Source code repository
    Repository name URI
    Reproducible Research Repository (World Bank) https://reproducibility.worldbank.org
    Software
    Name
    Python
    Version
    3.13.2

    Reproducibility

    Technology environment

    Paper exhibits were reproduced on a computer with the following specifications:
    • OS: Windows 11 Enterprise
    • Processor: INTEL(R) XEON(R) PLATINUM 8562Y+ (2.80 GHz) (4 processors)
    • Memory available: 32.0 GB
    Technology requirements

    Run time: ~ 1 minute
    Reproduction instructions

    To reproduce the findings in this paper, a replicator must:

    1. Restore the environment in requirements.txt.
    2. Run the script scripts/run_analysis.py
      Refer to the README for detailed instructions.

    Data

    Datasets
    Name
    IEG Data: World Bank Project Lessons
    Note
    Data accessed on March 12, 2025. Contains lessons learned text and evaluation metadata from IEG's Implementation Completion Report Reviews (ICRRs) and Project Performance Assessment Reports (PPARs), covering World Bank lending projects evaluated from approximately 1995 to the present. Files: data/raw/IEG_ICRR-PPAR_Lessons_2025-03-12.xlsx; data/raw/IEG_ICRR-PPAR_Lessons_2025-03-12_patches.csv.
    Access policy
    Data is publicly available and included in the reproducibility package.
    License
    Creative Commons Attribution 4.0 International (CC BY 4.0)
    License URL
    https://www.worldbank.org/en/about/legal/terms-of-use-for-datasets
    Data URL
    https://ieg.worldbankgroup.org/sites/default/files/Data/IEG_ICRR-PPAR_Lessons_2025-03-12.xlsx
    Citation
    Independent Evaluation Group (IEG), World Bank Group. 2025. IEG Data: World Bank Project Lessons [dataset]. File: IEG_ICRR-PPAR_Lessons_2025-03-12.xlsx. Retrieved March 12, 2025, from https://ieg.worldbankgroup.org/page/ieg-data-world-bank-project-lessons
    Name
    Harmonized List of Fragile and Conflict-Affected Situations (FCS)
    Note
    Data accessed in January 2026. World Bank Harmonized List of Fragile and Conflict-affected Situations (FCS), published annually. Used to classify projects by the FCS status of the borrowing country for fiscal years 2015–2025. Files: data/reference/FCS/FCS.xlsx.
    Access policy
    Data is publicly available and included in the reproducibility package.
    License
    Creative Commons Attribution 4.0 International (CC BY 4.0)
    License URL
    https://www.worldbank.org/en/about/legal/terms-of-use-for-datasets
    Data URL
    https://www.worldbank.org/en/topic/fragilityconflictviolence/brief/harmonized-list-of-fragile-situations
    Citation
    World Bank Group. 2025. Harmonized List of Fragile and Conflict-Affected Situations: FY06 to FY25 [dataset]. Retrieved January 2026, from https://www.worldbank.org/en/topic/fragilityconflictviolence/brief/harmonized-list-of-fragile-situations
    Name
    Innovation Keyword Dictionaries
    Note
    Author-constructed JSON keyword dictionaries used for innovation tagging and taxonomy classification. Developed using a combination of LLM assistance and expert review (see methodology/ folder for prompt records and review protocols). Files: data/reference/keywords/*.json. Files, pipeline steps, and LLMs used: (1) innovation_keywords.json - Step 2, ChatGPT (GPT-4o) and DeepSeek (V3); (2) taxonomy_keywords.json - Step 7, ChatGPT (GPT-4o) and DeepSeek (V3); (3) pilot_scale_keywords.json, (4) ifc_keywords.json, and (5) pcm_keywords.json - Step 10, ChatGPT (GPT-4o); (6) top_models_keywords.json - Step 15, ChatGPT (GPT-5) and DeepSeek (V3). All developed June-October 2025.
    Access policy
    Data is publicly available and included in the reproducibility package.
    Citation
    Authors' compilation. 2025. "Innovation Keyword Dictionaries" [dataset]. Constructed using LLM assistance and expert review for the analysis of innovation patterns in World Bank Group project evaluations
    Data statement

    All data sources are publicly available and included in the reproducibility package.

    Description

    Output
    Type
    Working Paper
    Title
    Innovation Patterns In The World Bank Group Portfolio
    Description
    Policy Research Working Papers (PRWP)
    Authors
    Author Affiliation Email
    Natalia Agapitova World Bank Group nagapitova@worldbank.org
    Anastasia Nedayvoda World Bank Group anedayvoda@ifc.org
    Stephen Winkler World Bank Group swinkler2@worldbank.org
    Amschel de Rothschild World Bank Group aderothschild@worldbank.org
    Ergun Ertekin World Bank Group eertekin@ifc.org
    Sarah Lenoble World Bank Group slenoble@worldbank.org
    Date of production

    2026-05-01

    Scope and coverage

    Geographic locations
    Location Code
    World WLD
    Topics
    ID Topic Parent topic ID Vocabulary Vocabulary URI
    C45 Neural Networks and Related Topics C4 Journal of Economic Literature (JEL)
    O19 International Linkages to Development • Role of International Organizations O1 Journal of Economic Literature (JEL)
    C55 Large Data Sets: Modeling and Analysis C5 Journal of Economic Literature (JEL)
    O31 Innovation and Invention: Processes and Incentives O3 Journal of Economic Literature (JEL)

    Disclaimer

    Disclaimer

    The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development/The World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.

    Access and rights

    License
    Name URI
    Modified BSD3 https://opensource.org/license/bsd-3-clause/

    Contacts

    Contacts
    Name Affiliation Email
    Natalia Agapitova World Bank nagapitova@worldbank.org
    Reproducibility WBG World Bank reproducibility@worldbank.org

    Information on metadata

    Producers
    Name Abbreviation Affiliation Role
    Reproducibility WBG DECDI World Bank - Development Impact Department Verification and preparation of metadata
    Date of Production

    2026-05-01
    Document version

    1

    Citation

    Citation
    loading, please wait...
    Export citation: RIS | BibTeX | Plain text
    Back to Catalog