{"type":"script","doc_desc":{"producers":[{"name":"Reproducibility WBG","abbr":"DECDI","affiliation":"World Bank - Development Impact Department","role":"Verification and preparation of metadata"}],"prod_date":"2025-12-18","version":"1"},"project_desc":{"authoring_entity":[{"name":"Koen Geven","affiliation":"World Bank","email":"kgeven@worldbank.org"},{"name":"Ayesha Tahir","affiliation":"World Bank","email":"atahir@worldbank.org"},{"name":"Tazeen Fasih","affiliation":"World Bank","email":"tfasih@worldbank.org"},{"name":"Javaeria Qureshi","affiliation":"University of Illinois at Chicago","email":"javaeria@uic.edu"},{"name":"Amer Hasan","affiliation":"World Bank","email":"ahasan1@worldbank.org"},{"name":"Sheena Fazili","affiliation":"Independent Researcher","email":"fazilisheena@gmail.com"},{"name":"Rabea Malik","affiliation":"IDEAS Pakistan","email":"rabea.malik@ideaspak.org"},{"name":"Kevin MacDonald","affiliation":"Independent Researcher","email":"kadmacdonald@gmail.com"}],"title_statement":{"title":"Reproducibility package for Nudging At Scale: Evidence From A Government Text Messaging Campaign During School Shutdowns In Punjab, Pakistan","idno":"RR_PAK_2025_483"},"data_statement":"Some data is restricted and has not been included in the reproducibility package.","software":[{"name":"Stata","version":"18.5 MP"}],"scripts":[{"title":"Reproducibility package for Nudging At Scale: Evidence From A Government Text Messaging Campaign During School Shutdowns In Punjab, Pakistan","date":"2025-12","notes":"Computational reproducibility verified by Development Impact (DECDI) Analytics team, World Bank.","instructions":"See README in reproducibility package.","file_name":"RR_PAK_2025_483","zip_package":"RR_PAK_2025_483.zip","dependencies":"Stata dependencies are listed in the ado folder."}],"repository_uri":[{"name":"Reproducible Research Repository (World Bank)","uri":"https:\/\/reproducibility.worldbank.org"}],"production_date":"2025-12-18","abstract":"Text and voice messages have emerged as a low cost and popular tool to nudge recipients to change behavior. This paper presents findings from a randomized control trial (RCT) designed to evaluate the impact of an information campaign using text and voice messages implemented in Punjab, Pakistan during the COVID-19-induced school closures. This campaign sought to increase study time and provide academic support while schools were closed and to encourage re-enrollment when they opened to reduce dropout. The campaign targeted girls enrolled in grades 5-7. Messages were sent out by a government institution, and the campaign lasted from October, 2020 until November, 2021, when schools had permanently re-opened. Households were randomized across three treatment groups and a control group that did not receive any messages. The first treatment group received gender-specific messages that explicitly referenced daughters in their households, whereas the second treatment group received gender-neutral messages. A third group was cross-randomized across the first two treatment arms and received academic support messages (practice math problems and solutions). Results show that messages increased re-enrollment by 6.0 percentage points approximately three months after the intervention finished. The message program also increased learning outcomes by 0.2 S.D. for Urdu and 0.2 S.D. for math. The paper also finds a small positive effect on the intensive margin of remote learning and an (equivalent) small negative effect on the intensive margin of outside tutoring. In line with similar studies on pandemic remediation efforts, the paper finds no effect of the academic support intervention on learning. The findings suggest that increased school enrolment played a large role in supporting the observed increase in learning outcomes.","geographic_units":[{"name":"Pakistan","code":"PAK"}],"keywords":[{"name":"Education"},{"name":"Gender"},{"name":"Text Messages"},{"name":"Covid-19"},{"name":"Parenting"}],"topics":[{"id":"C93","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Field Experiments","parent_id":"C9"},{"id":" I21","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Analysis of Education","parent_id":"I2"},{"id":" O15","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Human Resources \u2022 Human Development \u2022 Income Distribution \u2022 Migration","parent_id":"O1"},{"id":" J16","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Economics of Gender \u2022 Non-labor Discrimination","parent_id":"J1"}],"output":[{"type":"Working Paper","description":"Policy Research Working Papers (PRWP)","title":"Nudging At Scale: Evidence From A Government Text Messaging Campaign During School Shutdowns In Punjab, Pakistan"}],"language":[{"name":"English","code":"EN"}],"technology_requirements":"Runtime: 30 minutes.","disclaimer":"The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development\/The World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.","license":[{"name":"Modified BSD3","uri":"https:\/\/opensource.org\/license\/bsd-3-clause\/"}],"contacts":[{"name":"Koen Geven","affiliation":"World Bank","email":"kgeven@worldbank.org"},{"name":"Reproducibility WBG","affiliation":"World Bank","email":"reproducibility@worldbank.org"}],"datasets":[{"name":"Survey data on school re-enrollment, assessment scores, and time use for girls in Punjab, Pakistan","note":"Source: data was collected by the authors. The final datasets were accessed in October 2025.\nData files: \"panel_data_R3_with012.dta\", \"panel_data_all.dta\".","access_type":"Data is forthcoming in the World Bank Microdata Library.","citation":"Geven, K., Tahir, A., Fasih, T., Qureshi, J., Hasan, A., Fazili, S., Malik, R., MacDonald, K.\n2025.\n\"Survey data on school re-enrollment, assessment scores, and time use for girls in Punjab, Pakistan [dataset]\".\nForthcoming in the World Bank Microdata Library.\nAccessed October 2025."}],"technology_environment":"Paper exhibits were reproduced on a computer with the following specifications:\n\u2022 OS: Windows 11 Enterprise, version 24H2\n\u2022 Processor: Intel(R) Core(TM) Ultra 7 165U (2.10 GHz)\n\u2022 Memory available: 32 GB","reproduction_instructions":"**1. Access the data:** All data used in the reproducibility package are forthcoming in the World Bank Microdata Library. Users need to gain access to the data to reproduce the results.\n**2. Run the code:** After data are accessed, open the do-file \"replication_file_00_master_sms_girl\", change the root folder path in line 17, and run the do-file.\n\nSince no data are included, the package includes the results produced by the replicators. These files can be used to review the results presented in the paper."},"tags":[{"tag":"DOI"},{"tag":"Forthcoming data"},{"tag":"Open Code"}],"schematype":"script"}