About

The Reproducible Research Repository is a one-stop shop for reproducibility packages associated with research authored by World Bank staff and consultants. To enhance the credibility, transparency, and impact of World Bank research, this platform publishes reproducibility packages that fully document the data and code on which research findings are based. The World Bank is the leading producer of development economics research. Publishing reproducibility packages allow all consumers of research to fully scrutinize the data and methodological choices underlying research results. Sharing data and code also democratizes development research by allowing others to replicate, build on, and extend research findings. Together, scrutiny and sharing improve the credibility, transparency and ultimately the impact of World Bank research.

The Reproducible Research Repository contains three collections: Policy Research Working Papers, for all papers published in the World Bank’s working paper series; Journal Articles, for academic journal articles authored by World Bank staff or consultants, and World Bank Reports, for flagship reports and other analytical outputs.

To ensure the credibility of the Reproducible Research Repository, reproducibility packages are all verified to ensure that they are complete and fully functional before they are published to the repository. Packages in the PRWP collection are all verified by the World Bank staff managing the Reproducible Research Repository. Packages in the Journal Articles are published as-is if verified by the journal prior to publication. Packages that are not verified by the journal are verified internally prior to publication in this repository. All packages that have been verified by the World Bank team include a Reproducibility Certificate which provides details on the scope of the verification and the reproducibility status of the package.

Reproducibility Verification

The World Bank reproducibility verification process includes the following checks:

• Descriptive statistics, number of observations, regression point estimates and standard errors are consistent across the outputs produced when the reviewer runs the code and the outputs presented in the manuscript.
• The graphs and visualizations produced by the reviewer are consistent with the outputs provided in the manuscript. This includes checking that the visualized data, x and y-titles, x and y-ticks, and the graphs legends are consistent.
• The outputs produced by the reviewer are stable; there are no changes across multiple runs of the code.
• The code is publication-ready; it does not contain any identifying information or comments flagging pending issues or conditions to verify.
• The code produces only the outputs included in the manuscript and associated appendices.
• The code produces all statistical tables, data visualization, and any in-text numeric references not directly drawn from tables.
• The input data provided directly by the authors matches exactly to the data that is described in the Data Accessibility Statement (this check is only performed if the review team can access the original data).

How to submit a reproducibility package

The Reproducible Research Repository hosts reproducibility packages for research and analytical work produced by World Bank staff and consultants. World Bank staff and consultants who would like to submit a package should follow these instructions:

• To submit a reproducibility package for a Policy Research Working Paper, click here for instructions
• To submit a reproducibility package for a Published Journal Article:
  • If your article has been published in a journal that requires a verified reproducibility package, please email a link to the published reproducibility package to reproducibility@worldbank.org and the team will create a repository entry. For a list of journals with verified reproducibility requirements, click here.
  • If your article has been published in a journal that does not require a verified reproducibility package, please click here and follow the same steps as for a Policy Research Working Paper
• To submit a reproducibility package for a World Bank Report, please email reproducibility@worldbank.org.

Disclaimer

The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development/the World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.

Questions about the contents of specific packages should be directed to the corresponding authors; contact details are provided under each catalogue entry. General questions should be directed to reproducibility@worldbank.org.