{"type":"script","doc_desc":{"producers":[{"name":"Reproducibility WBG","abbr":"DECDI","affiliation":"World Bank - Development Impact Department","role":"Verification and preparation of metadata"}],"prod_date":"2025-11-20","version":"1"},"project_desc":{"authoring_entity":[{"name":"Philip Wollburg","affiliation":"World Bank","email":"pwollburg@worldbank.org"},{"name":"Pauline Castaing","affiliation":"World Bank","email":"pcastaing@worldbank.org"},{"name":"Ivette Contreras","affiliation":"World Bank","email":"icontreras@worldbank.org"},{"name":"Yannick Markhof","affiliation":"ETH Z\u00fcrich, World Bank","email":"ymarkhof@ethz.ch"},{"name":"Akiko Sagesaka","affiliation":"World Bank","email":"asagesaka@worldbank.org"},{"name":"Amparo Palacios-Lopez","affiliation":"World Bank","email":"apalacioslopez@worldbank.org"}],"title_statement":{"title":"Reproducibility package for The Effect Of Survey Mode On Data Quality: Experimental Evidence From Nigeria","idno":"RR_NGA_2025_486"},"data_statement":"Data is forthcoming in the World Bank Microdata Library, and is not included in the reproducibility package. Some data is restricted and not included in the reproducibility package. For more details, please refer to the individual data entries and the README file.","software":[{"name":"Stata","version":"18.5"},{"name":"Excel"}],"scripts":[{"title":"Reproducibility package for The Effect Of Survey Mode On Data Quality: Experimental Evidence From Nigeria","date":"2025-11","notes":"Computational reproducibility verified by Development Impact (DECDI) Analytics team, World Bank.","instructions":"See README in reproducibility package.","file_name":"RR_NGA_2025_486","zip_package":"RR_NGA_2025_486.zip","dependencies":"Stata dependencies are listed in the ado folder."}],"repository_uri":[{"name":"Reproducible Research Repository (World Bank)","uri":"https:\/\/reproducibility.worldbank.org"}],"production_date":"2025-11-20","abstract":"In a large-scale experiment in rural Nigeria, we study the role of survey mode \u2014 in-person vs. over the phone \u2014 for survey measurement and data quality. Our experimental design isolates mode effects from other common sources of errors in surveys and covers twenty outcome measures across topics such as health, labor, shocks, wellbeing, and food security. We consistently find mode effects across outcomes, with phone responses differing from in-person responses by up to 63%, and a median difference of 17\u201318%. We document that these effects are large relative to other errors in phone surveys such as undercoverage of households without phones. Using a within-respondent design, we measure the full, respondent-level distribution of mode effects and find them to vary much more than averages let on. Respondents with higher education levels are less prone to mode effects whereas mode effects sharply increase in prevalence as respondents face more answer options. We present evidence that mode effects are driven by lower cognitive engagement of respondents over the phone. Given the growing reliance on phone surveys in LMICs, our findings provide novel guidance for survey design and advance the research frontier on how survey mode influences measurement.","geographic_units":[{"name":"Nigeria","code":"NGA"}],"keywords":[{"name":"Data Quality"},{"name":"Survey Data"},{"name":"Phone Survey"},{"name":"Measurement"},{"name":"Survey Mode"}],"topics":[{"id":"C81","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Methodology for Collecting, Estimating, and Organizing Microeconomic Data \u2022 Data Access","parent_id":"C8"},{"id":" C83","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Survey Methods \u2022 Sampling Methods","parent_id":"C8"},{"id":" O12","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Microeconomic Analyses of Economic Development","parent_id":"O1"}],"output":[{"type":"Working Paper","description":"Policy Research Working Papers (PRWP)","title":"The Effect Of Survey Mode On Data Quality: Experimental Evidence From Nigeria"}],"language":[{"name":"English","code":"EN"}],"technology_requirements":"The code takes approximately 5 minutes to run.","disclaimer":"The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development\/The World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.","license":[{"name":"Modified BSD3","uri":"https:\/\/opensource.org\/license\/bsd-3-clause\/"}],"contacts":[{"name":"Philip Wollburg","affiliation":"The World Bank","email":"pwollburg@worldbank.org"},{"name":"Reproducibility WBG","affiliation":"World Bank","email":"reproducibility@worldbank.org"}],"datasets":[{"uri":"https:\/\/microdata.worldbank.org\/","name":"General Household Survey, Panel 2023-2024 \u2013 Mixed-Mode Study ","access_type":"Data is forthcoming in the World Bank Microdata Library.","note":"Files location: Data\/Raw\/Mix mode study.\nDataset names: MM5, MM4, MM3, MM2, MM1, mm5_interviewers. mm5_interviwers.dta won't be published due to sensitive information. Datasets are not included in the reproducibility package. Because `mm5_interviewers.dta` is restricted, replicators will not be able to generate interviewer characteristics. For example, this will block execution of the code in `Part1_MasterDataCleaning.do` starting at line 223.","citation":"National Bureau of Statistics (NBS). (2024). \"General Household Survey, Panel 2023-2024 \u2013 Mixed-Mode Study\" [dataset]. World Bank, Development Data Group. Forthcoming on DDH."},{"name":"Nigeria's General Household Survey Panel Wave 5 (GHS-Panel Wave 5)","uri":"https:\/\/microdata.worldbank.org\/index.php\/catalog\/6410\/study-description","access_type":"Publicly available and included in the reproducibility package. ","license":"Creative Commons Attribution 4.0","license_uri":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/","citation":"National Bureau of Statistics (NBS). (2024). \"General Household Survey, Panel 2023-2024\" [dataset]. World Bank, Development Data Group. https:\/\/doi.org\/10.48529\/ZD5S-TJ25","note":"Access date: December 2024.\nFiles location: Data\/Raw\/GHS-Panel W5\/Post Planting Visit and Data\/Raw\/GHS-Panel W5\/Post Harvest Visit.\nAggregated consumption expenditure data (Data\/Raw\/GHS-Panel W5\/totcons.dta) and on languages used for the interview (Data\/Raw\/GHS-Panel W5\/secta_harvestw5 lang data.dta) will be forthcoming in the coming months in the microdata library and are not included in the reproducibility package. The data from the enumerators\u2019 questionnaire (Data\/Raw\/GHS-Panel W5\/GHS-Panel interviewers surveys) will not be released due to sensitivity concerns and is not included in the reproducibility package. Because GHS-Panel interviewers survey data is restricted, replicators will not be able to generate interviewer characteristics. For example, this will block execution of the code in `Part1_MasterDataCleaning.do` starting at line 680."},{"name":"Living Standards Measurement Study - High Frequency Phone Surveys (LSMS-HFPS) Harmonized Dataset","uri":"https:\/\/github.com\/lsms-worldbank\/HFPS_harmonization_working\/blob\/main\/Final%20datasets\/HH_%204%20Dec%202024.zip ","note":"Access date: December 2024.\nFiles location: Data\/Raw\/HFPS.","access_type":"Publicly available and included in the reproducibility package. ","citation":"World Bank LSMS \/ HFPS Harmonization Working Group. (2024). \"HH_ 4 Dec 2024\" [dataset]. LSMS-World Bank GitHub repository. ","license":"Creative Commons Attribution 4.0","license_uri":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"reproduction_instructions":"To run the package:\n1. **Acquire the Data**. Please refer to the *Datasets* subsection of this documentation and the README file for detailed information on how to access each dataset.\n2. **Open the Main Do-File**. Open the file `MasterDofile_ModeEffectsNigeria.do`, update the global directory path, and set your user number as indicated in the script.\n3. **Run the Code**. Execute the do-file from start to finish. This will run all components of the analysis and generate the outputs.","technology_environment":"Paper exhibits were reproduced on a computer with the following specifications:\n\u2022 OS: Windows 11 Enterprise\n\u2022 Processor: Intel(R) Xeon(R) Gold 5218 CPU @2.30GHz\n\u2022 Memory available: 6 GB\n\u2022 Software version: Stata 18.5 MP","reproducibility_status":{"note":"Because not all required datasets are currently publicly available, the package includes the results generated by the replicators. These files allow users to verify the results presented in the paper and compare them with the authors\u2019 outputs until full data access is possible."}},"tags":[{"tag":"Accessible Data"},{"tag":"DOI"},{"tag":"Open Code"}],"schematype":"script"}