Reproducibility package for Developing A Measure Of Women’s Agency In Tunisia: Machine Learning And Generative Artificial Intelligence Approaches

Women’s agency – i.e. women’s ability to define and act upon their goals – is often the target of policies and programs, both as a means of reducing other gender inequities and as an end in and of itself. Measuring agency is complex and variation in how the concept is operationalized contributes to difficulty in synthesizing evidence on the topic. At the same time, standard measures that are used in large-scale survey programs are often treated as universal and compared across widely differing settings. This paper investigates the extent to which a single scale for women’s agency can be used across contexts. This study replicates, and builds upon, a recent study by Jayachandran et al. (2023) in India in which the authors used qualitative data and machine learning (ML) techniques to develop a five-question measure of women’s agency. This paper applies a variety of ML and, in a new contribution, generative artificial intelligence (GenAI) techniques to qualitative in-depth interviews and quantitative surveys from the same sample of women in Tunisia to identify what questions best measure overall agency. The main finding is that context matters. Overlap between the questions that performed well in India and in Tunisia is minimal. Thebest agency questions selected by ML from one context (e.g., a rural area) do not perform well in other contexts (e.g., an urban area). Researchers may need to generate and use context-specific agency measures to accurately measure agency. Although ML approaches perform well against a benchmark qualitative agency score, GenAI ones do not. Current, readily available GenAI tools do not appear to have the capabilities needed to make this exercise more effective or less time-consuming.