{"type":"script","doc_desc":{"producers":[{"name":"Reproducibility WBG","abbr":"DECDI","affiliation":"World Bank - Development Impact Department","role":"Verification and preparation of metadata"}],"prod_date":"2026-10-08","version":"1"},"project_desc":{"authoring_entity":[{"name":"Patrick Premand","affiliation":"World Bank","email":"ppremand@worldbank.org"},{"name":"Edward Jee","affiliation":"Northwestern University","email":"edward.jee@kellogg.northwestern.edu"},{"name":"Dean Karlan","affiliation":"Northwestern University","email":"dean.karlan@gmail.com"},{"name":"Samantha Carter","affiliation":"Livelihood Impact Fund","email":"samanthaw.carter@gmail.com"}],"title_statement":{"title":"Reproducibility package for Doing Better At Doing Good: Lessons From A Bayesian Meta-Analysis After 20-Years Of Poverty Graduation Programs","idno":"RR_WLD_2026_756"},"data_statement":"The reproducibility package relies on 1 type of data: open. For the source, access type, file location, and citation of each dataset, refer to the Datasets section and the README file.","software":[{"name":"R","version":"4.5.3"},{"name":"Python","version":"3.11"},{"name":"CmdStan ","version":"2.36.0"},{"name":"Podman ","version":"6.1.2"}],"scripts":[{"title":"Reproducibility package for Doing Better At Doing Good: Lessons From A Bayesian Meta-Analysis After 20-Years Of Poverty Graduation Programs","date":"2026-10","notes":"Computational reproducibility verified by Development Impact (DECDI) Analytics team, World Bank.","instructions":"See README in reproducibility package.","file_name":"RR_WLD_2026_756","zip_package":"RR_WLD_2026_756.zip","dependencies":"Software dependencies are included in the package."}],"repository_uri":[{"name":"Reproducible Research Repository (World Bank)","uri":"https:\/\/reproducibility.worldbank.org"}],"production_date":"2026-10-08","abstract":"Multi-faceted poverty Graduation programs that aim to increase income generating activities for low-income households are being rapidly adopted and scaled by nonprofit organizations and governments. We conduct a Bayesian hierarchical meta-analysis of 49 studies (encompassing 81 treatment variations) on poverty Graduation programs, defined as containing at least three of five components: consumption support, savings facilitation, skills training, coaching, and a lump-sum transfer. We find large, sustained gains across economic and psychosocial outcomes: monthly consumption rises by 38 USD PPP (11%), income by 28 USD PPP (15%), alongside improvements in savings, assets, food security, mental health, and women\u2019s empowerment. Beyond these average treatment effects, three particular results stand out. First, programs with large transfers (at least 400 USD PPP) have larger treatment effects but are not more cost-effective. Second, more expensive programs generate larger treatment effects, but the improvements are not commensurate with the cost increases; thus, lower cost programs tend to be more cost-effective. Third, programs implemented 2016 onward perform considerably better than those implemented prior to 2016. Benefit-cost ratios from 2016 onward are 4.85, compared to 2.88 through 2015. Neither changes in cost nor in high-level design (i.e., components) explain this upward trend. Considering other heavily-studied interventions, we find a similar upward trend for treatment effects on behavior change from financial education but not for unconditional cash transfers. We hypothesize a mechanism of continuous sector-community learning and improved implementation quality.","geographic_units":[{"name":"World","code":"WLD"}],"output":[{"type":"Working Paper","description":"Policy Research Working Paper (PRWP)","title":"Doing Better At Doing Good: Lessons From A Bayesian Meta-Analysis After 20-Years Of Poverty Graduation Programs"}],"language":[{"name":"English","code":"EN"}],"technology_requirements":"Run time: ~ 1.5 hours","disclaimer":"The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development\/The World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.","license":[{"name":"MIT License","uri":"https:\/\/opensource.org\/license\/mit"},{"name":"World Bank IGO Rider","uri":"https:\/\/github.com\/worldbank\/metadata-editor\/blob\/main\/WB-IGO-RIDER.md"}],"contacts":[{"name":"Patrick Premand","affiliation":"World Bank","email":"ppremand@worldbank.org"},{"name":"Reproducibility WBG","affiliation":"World Bank","email":"reproducibility@worldbank.org"}],"datasets":[{"note":"Authors' compilation of treatment effects, uncertainty, program components, covariates and costs extracted from 53 randomized evaluations of poverty graduation programs, prepared through the DEVIDENCE process on 23 September 2026. Individual study citations and links are retained in studies.csv. File locations: data\/analysis\/graduation_effects.csv (5,097 effect estimates); data\/analysis\/graduation_arms.csv (99 treatment arms); data\/analysis\/studies.csv (53 studies); data\/analysis\/manifest.json (checksums, columns, units and keys). A variant of these three tables used only for the included-study table is at data\/study-list\/. Raw extraction files, coding workbooks and preparation code are not included. The package also supplies author-generated intermediate results (posterior draws, model fits and summaries) in data\/derived\/; these are produced from this database by upstream steps that are not fully rerun in the package.\n","name":"Graduation Meta-Analysis Dataset","access_type":"Data is included in the reproducibility package and also forthcoming to World Bank Development Data Hub. ","citation":"Authors' compilation. 2026. \"Graduation Meta-Analysis Dataset\" [dataset]. Based on published studies of Graduation programs listed in studies.csv, prepared through the DEVIDENCE process by the Graduation meta-analysis contributors, 23 September 2026"},{"note":"File location: data\/analysis\/cash_effects.csv (646 rows, 152 columns). Estimate-level dataset from the Crosta et al. meta-analysis of unconditional cash transfers: treatment-effect estimates extracted from 113 source papers (72 programs, 34 countries, publication years 2006 to 2024), with the estimates as reported and in harmonized form, study metadata, and external GDP and poverty data. Used as the cash comparison in the paper (effects per $100 transferred). The file retains the source-paper identifiers, citations and URLs and adds a source-row identifier. The dataset has no title of its own in the package, so it is cited through the working paper that documents it. The compilation was accessed on 30 August 2026; the working paper revision date (August 2026) was checked on 11 September 2026 and does not establish the compilation version. The contributors\u2019 additions are licensed CC BY 4.0, with source citations retained; this does not relicense the underlying papers or the Crosta et al. work, and no license is stated for the latter. Redistribution of the supplied final table is authorized.\n    ","name":"Unconditional Cash Transfers (Crosta et al.)","citation":"Crosta, T., Karlan, D., Ong, F., R\u00fcschenp\u00f6hler, J., and Udry, C. R. 2026. \u201cUnconditional Cash Transfers: A Bayesian Meta-Analysis of Randomized Evaluations in Low and Middle Income Countries\u201d [dataset]. NBER Working Paper 32779. https:\/\/doi.org\/10.3386\/w32779. Accessed August 2026.   ","access_type":"Data is included in the reproducibility package."},{"note":"File location: data\/analysis\/financial_literacy_effects.csv (673 rows). Adapted by the contributors from the Kaiser et al. replication dataset (Mendeley Data, version 2): selected fields exported to CSV, original row order preserved with a row identifier, and implementation-year coding, evidence and source links added. The analysis constructs scaled features from numeric codes. Downloaded on 30 August 2026.","name":"Replication code and data for 'Financial education affects financial knowledge and downstream behaviors' (by Tim Kaiser, Annamaria Lusardi, Lukas Menkhoff and Carly Urban)","access_type":"Data is included in the reproducibility package.","citation":"Kaiser, Tim. 2021. \"Replication code and data for 'Financial education affects financial knowledge and downstream behaviors' (by Tim Kaiser, Annamaria Lusardi, Lukas Menkhoff and Carly Urban)\" [dataset]. Mendeley Data, V2. https:\/\/doi.org\/10.17632\/svc3t8v3hs.2. Accessed August 2026.\n","license_uri":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/","uri":"https:\/\/data.mendeley.com\/datasets\/svc3t8v3hs\/2","license":"Creative Commons Attribution 4.0 International (CC BY 4.0)"}],"reproduction_instructions":"The findings of the paper can be reproduced after:\n\n1. Starting the Podman machine using podman machine start.\n2. Building the container using podman build --platform linux\/amd64 -f containers\/Containerfile -t graduation-replication .\n3. Running the run-replication script inside the container following the README instructions.","technology_environment":"Paper exhibits were reproduced on a computer with the following specifications:\n\u2022 OS: macOS 26.5.1\n\u2022 Processor: Apple M5 (10-core CPU)\n\u2022 Memory available: 16.0 GB"},"datacite":{"creators":[{"givenName":"Patrick","familyName":"Premand","nameType":"Personal","affiliation":[{"name":"World Bank","affiliationIdentifier":"https:\/\/ror.org\/00ae7jd04","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]},{"givenName":"Edward","familyName":"Jee","nameType":"Personal","affiliation":[{"name":"Northwestern University","affiliationIdentifier":"https:\/\/ror.org\/000e0be47","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]},{"givenName":"Dean","familyName":"Karlan","nameType":"Personal","affiliation":[{"name":"Northwestern University","affiliationIdentifier":"https:\/\/ror.org\/000e0be47","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]},{"givenName":"Samantha","familyName":"Carter","nameType":"Personal","affiliation":[{"name":"Livelihood Impact Fund"}]}],"titles":[{"lang":"en","title":"Reproducibility package for Doing Better At Doing Good: Lessons From A Bayesian Meta-Analysis After 20-Years Of Poverty Graduation Programs"},{"title":"RR_WLD_2026_756","titleType":"Other"}],"publisher":"World Bank","publicationYear":"2026","types":{"resourceType":"Reproducibility package","resourceTypeGeneral":"Other"},"url":"https:\/\/reproducibility.worldbank.org\/index.php\/catalog\/study\/RR_WLD_2026_756","language":"en"},"tags":[{"tag":"DOI"},{"tag":"Open Code"},{"tag":"Open Data"}],"schematype":"script"}