{"type":"script","doc_desc":{"producers":[{"name":"Reproducibility WBG","abbr":"DECDI","affiliation":"World Bank - Development Impact Department","role":"Verification and preparation of metadata"}],"prod_date":"2026-08-21","version":"1"},"project_desc":{"authoring_entity":[{"name":"Caroline Krafft","affiliation":"Humphrey School of Public Affairs, University of Minnesota","email":"kraff004@umn.edu"},{"name":"Leila Baghdadi","affiliation":"World Bank","email":"lbaghdadi@worldbank.org"},{"name":"Roberta Gatti","affiliation":"World Bank","email":"rgatti@worldbank.org"},{"name":"Asif M. Islam","affiliation":"World Bank","email":"aislam@worldbank.org"},{"name":"Maia Sieverding","affiliation":"World Bank","email":"msieverding@worldbank.org"}],"title_statement":{"title":"Reproducibility package for Developing A Measure Of Women\u2019s Agency In Tunisia: Machine Learning And Generative Artificial Intelligence Approaches","idno":"RR_TUN_2026_705"},"data_statement":"Some data is not yet publicly available but is expected to be made available through the World Bank Microdata Library in the future","software":[{"name":"R","version":"4.5.2"},{"name":"Stata","version":"19.5 MP"}],"scripts":[{"title":"Reproducibility package for Developing A Measure Of Women\u2019s Agency In Tunisia: Machine Learning And Generative Artificial Intelligence Approaches","date":"2026-08","notes":"Computational reproducibility verified by Development Impact (DECDI) Analytics team, World Bank.","instructions":"See README in reproducibility package.","file_name":"RR_TUN_2026_705","zip_package":"RR_TUN_2026_705.zip","dependencies":"R dependencies are listed in the file renv.lock. Stata dependencies are listed in the ado folder."}],"repository_uri":[{"name":"Reproducible Research Repository (World Bank)","uri":"https:\/\/reproducibility.worldbank.org"}],"production_date":"2026-08-21","abstract":"Women\u2019s agency \u2013 i.e. women\u2019s ability to define and act upon their goals \u2013 is often the target of policies and programs, both as a means of reducing other gender inequities and as an end in and of itself. Measuring agency is complex and variation in how the concept is operationalized contributes to difficulty in synthesizing evidence on the topic. At the same time, standard measures that are used in large-scale survey programs are often treated as universal and compared across widely differing settings. This paper investigates the extent to which a single scale for women\u2019s agency can be used across contexts. This study replicates, and builds upon, a recent study by Jayachandran et al. (2023) in India in which the authors used qualitative data and machine learning (ML) techniques to develop a five-question measure of women\u2019s agency. This paper applies a variety of ML and, in a new contribution, generative artificial intelligence (GenAI) techniques to qualitative in-depth interviews and quantitative surveys from the same sample of women in Tunisia to identify what questions best measure overall agency. The main finding is that context matters. Overlap between the questions that performed well in India and in Tunisia is minimal. Thebest agency questions selected by ML from one context (e.g., a rural area) do not perform well in other contexts (e.g., an urban area). Researchers may need to generate and use context-specific agency measures to accurately measure agency. Although ML approaches perform well against a benchmark qualitative agency score, GenAI ones do not. Current, readily available GenAI tools do not appear to have the capabilities needed to make this exercise more effective or less time-consuming. ","geographic_units":[{"name":"Tunisia","code":"TUN"}],"keywords":[{"name":"Gender"},{"name":"Agency"},{"name":"Empowerment"},{"name":"Qualitative Methods"},{"name":"Machine Learning"},{"name":"Artificial Intelligence"}],"topics":[{"id":"J16","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Economics of Gender \u2022 Non-labor Discrimination","parent_id":"J1"},{"id":" C81","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Methodology for Collecting, Estimating, and Organizing Microeconomic Data \u2022 Data Access","parent_id":"C8"},{"id":" C83","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Survey Methods \u2022 Sampling Methods","parent_id":"C8"},{"id":" C52","uri":"https:\/\/www.aeaweb.org\/econlit\/jelCodes.php?view=jel","vocabulary":"Journal of Economic Literature (JEL)","name":"Model Evaluation, Validation, and Selection","parent_id":"C5"}],"output":[{"type":"Working Paper","description":"Policy Research Working Papers (PRWP)","title":"Developing A Measure Of Women\u2019s Agency In Tunisia: Machine Learning And Generative Artificial Intelligence Approaches"}],"language":[{"name":"English","code":"EN"}],"technology_requirements":"Run time ~ 39 hours","disclaimer":"The materials in the reproducibility packages are distributed as they were prepared by the staff of the International Bank for Reconstruction and Development\/The World Bank. The findings, interpretations, and conclusions expressed in this event do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank, or the governments they represent. The World Bank does not guarantee the accuracy of the materials included in the reproducibility package.","license":[{"name":"MIT License","uri":"https:\/\/opensource.org\/license\/mit"},{"name":"World Bank IGO Rider","uri":"https:\/\/github.com\/worldbank\/metadata-editor\/blob\/main\/WB-IGO-RIDER.md"}],"contacts":[{"name":"Caroline Krafft","affiliation":"Humphrey School of Public Affairs, University of Minnesota","email":"kraff004@umn.edu"},{"name":"Reproducibility WBG","affiliation":"World Bank","email":"reproducibility@worldbank.org"}],"datasets":[{"name":"Tunisia Multiple Indicator Cluster Survey (MICS) 2023","note":"Data accessed December 19, 2024. The 2023 Tunisia MICS household roster data used for validation. Data is publicly available to registered users from the MICS UNICEF portal. Download the files and copy hl.sav (the household data) into the designated folder. File location: data\/original data\/hl.sav.","access_type":"Data is publicly available but does not allow redistribution and is not included in the reproducibility package.","license":"Custom Licence","uri":"https:\/\/mics.unicef.org\/surveys","citation":"Institut National Statistique (INS) and UNICEF. 2023. \"Tunisia Multiple Indicator Cluster Survey 2023\" [dataset]. Tunis: Institut National Statistique (INS) and UNICEF. https:\/\/mics.unicef.org\/surveys. Accessed December 2024."},{"name":"Women's Agency in Tunisia - Quantitative Survey and Manually Coded Qualitative Scores","note":" The main, anonymized data file serving as the starting point for the public replication. Contains quantitative survey data and manually coded qualitative scores. Additional data files are created through the do-files during replication (e.g., TN WAM PUBLIC analysis.dta, which is the main dataset used for analyses). File location: data\/analysis data\/TN WAM PUBLIC matched.dta.","access_type":"Data is forthcoming in the World Bank Microdata Library ","license":"Custom license","citation":"Krafft, C., Baghdadi, L., Gatti, R., Islam, A.M., Sieverding, M. Forthcoming. \"Developing a Measure of Women's Agency in Tunisia: Machine Learning and Generative Artificial Intelligence Approaches\" [dataset]. World Bank Policy Research Working Paper. Forthcoming to the World Bank Microdata Library. "},{"name":"AI Scores - Claude Haiku 4.5 Generated Scores (April 17, 2026)","note":"Generated using Claude Haiku 4.5 on April 17, 2026, using an institutional license through the web interface. File location: data\/original data\/LowAI_Claude_allscores_17Apr26.xlsx.","access_type":"Data is included in the package","license":"Custom License","citation":"Authors' compilation. 2026. \"AI Scores - Claude Haiku 4.5 Generated Scores\" [dataset]. Generated on April 17, 2026."},{"name":"AI Domains - Claude Haiku 4.5 Generated Domain Scores (April 20, 2026)","note":"Generated using Claude Haiku 4.5 on April 20, 2026, using an institutional license through the web interface. File location: data\/original data\/Low-mid AI_Claude_all scores_20Apr26.xlsx.","access_type":"Data is included in the package","license":"Custom License","citation":"Authors' compilation. 2026. \"AI Domains - Claude Haiku 4.5 Generated Domain Scores\" [dataset]. Generated on April 20, 2026 "},{"name":"Tunisia WAM Survey Variable Text","note":"Contains text for the questions asked in the Women's Agency in Tunisia survey. Author-compiled auxiliary file included in the replication package. File location: data\/original data\/TN WAM variable text.xlsx.","access_type":"Data is included in the package","license":"Custom License","citation":"Authors' compilation. 2026. \"Tunisia WAM Survey Variable Text\" [dataset]. Text for questions asked in the Women's Agency in Tunisia survey."},{"name":"AI Text-Only Questions Dataset","note":"Contains questions created by two AI methods using text generation. The AI qualitative questions portion was generated using NotebookLM (Google Gemini 3) via a web interface with an institutional license on March 10, 2026. The AI generated questions portion was produced using Google Gemini 3 via a web interface on March 10, 2026. File location: data\/original data\/AI text only.xlsx.","access_type":"Data is included in the package","license":"Custom License","citation":"Authors' compilation. 2026. \"AI Text-Only Questions Dataset\" [dataset]. Generated using NotebookLM (Google Gemini 3) and Google Gemini 3 via web interface with institutional license, Generated on March 10, 2026."},{"name":"Jayachandran et al. (2023)  - Using machine learning and qualitative interviews to design a five-question survey module for women\u2019s agency","note":"Data accessed from the paper available at https:\/\/doi.org\/10.1016\/j.worlddev.2022.106076. Contains variables selected by the Jayachandran et al. (2023) paper \"Using machine learning and qualitative interviews to design a five-question survey module for women\u2019s agency\", used in the Women's Agency in Tunisia study. File location: data\/original data\/JBC.dta.","access_type":"Data is publicly available and included in the reproducibility package.","license":"Open Access","uri":"https:\/\/doi.org\/10.1016\/j.worlddev.2022.106076","citation":"Jayachandran, S., Biradavolu, M., and Cooper, J. 2023. \"Using Machine Learning and Qualitative Interviews to Design a Five-Question Women's Agency Index\" [dataset]. World Development. https:\/\/doi.org\/10.1016\/j.worlddev.2022.106076.","license_uri":"https:\/\/www.sciencedirect.com\/journal\/information-and-organization\/publish\/open-access-options"}],"reproduction_instructions":"To reproduce the findings in this paper, a replicator must:\n1. Open the main do file `TN WAM Root v12 REP` update the directories on line 24, install the required packages and run the script  through line 103\n2. Open the  R script `TN WAM Random Forest v13 CK`,  update the directory on line 15, install the required packages and run the script \n3. Return to Stata do file `TN WAM Root v12 REP` and run lines 112 up to the end. ","technology_environment":"Paper exhibits were reproduced on a computer with the following specifications:\n\u2022 OS: Windows 11 Enterprise\n\u2022 Processor: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz (2.30 GHz) (2 processors)\n\u2022 Memory available: 16 GB"},"datacite":{"creators":[{"givenName":"Caroline","familyName":"Krafft","nameType":"Personal","affiliation":[{"name":"Humphrey School of Public Affairs, University of Minnesota","affiliationIdentifier":"https:\/\/ror.org\/017zqws13","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]},{"givenName":"Leila","familyName":"Baghdadi","nameType":"Personal","affiliation":[{"name":"World Bank","affiliationIdentifier":"https:\/\/ror.org\/00ae7jd04","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]},{"givenName":"Roberta","familyName":"Gatti","nameType":"Personal","affiliation":[{"name":"World Bank","affiliationIdentifier":"https:\/\/ror.org\/00ae7jd04","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]},{"givenName":"Asif M.","familyName":"Islam","nameType":"Personal","affiliation":[{"name":"World Bank","affiliationIdentifier":"https:\/\/ror.org\/00ae7jd04","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]},{"givenName":"Maia","familyName":"Sieverding","nameType":"Personal","affiliation":[{"name":"World Bank","affiliationIdentifier":"https:\/\/ror.org\/00ae7jd04","affiliationIdentifierScheme":"ROR","schemeUri":"https:\/\/ror.org"}]}],"titles":[{"lang":"en","title":"Reproducibility package for Developing A Measure Of Women\u2019s Agency In Tunisia: Machine Learning And Generative Artificial Intelligence Approaches"},{"title":"RR_TUN_2026_705","titleType":"Other"}],"publisher":"World Bank","publicationYear":"2026","types":{"resourceType":"Reproducibility package","resourceTypeGeneral":"Other"},"url":"https:\/\/reproducibility.worldbank.org\/index.php\/catalog\/study\/RR_TUN_2026_705","language":"en"},"tags":[{"tag":"DOI"},{"tag":"Forthcoming data"},{"tag":"Open Code"}],"schematype":"script"}