Reproducibility package for First Impressions Matter: Measuring And Correcting Position Bias In Business Environment Estimates

Estimating the constraints in the business environment guides development policy and allocation of resources. This paper examines response order bias in survey questions on the biggest obstacle to operations with a lengthy list of response options in the World Bank Enterprise Surveys. Using over 200,000 interviews with firm managers from 178 economies with randomized response orders, estimates show that the first three positions have a 9 percent higher relative likelihood of being selected, with the very first position on the showcard being 22 percent more likely to be selected. The primacy bias, tendency to select an early response option, is significantly stronger in low-income economies (17 percent, whereas in high-income economies the bias is statistically non-existent), creating a troubling paradox: position effects distort estimates most severely where accurate constraint measurement matters most for development policy. Applying empirical Bayes-stabilized inverse probability weights to correct for these biases changes the rank order for 9 percent of the obstacles, affecting about half of the surveys.