Reproducibility Package for Missing Evidence: Tracking Academic Data Use around the World

World Bank World Development Indicators (WDI)

File name: correlates_df.csv The file contains indicator data from the World Bank World Development Indicators (WDI). A set of indicators were pulled from the World Bank API, which was current as of November 2023. However, because the WDI is updated periodically, the data may have changed since the data were downloaded. Data was pulled using the web stats package in R in wdi_pull.R. Please refer to the README for the collections of indicators pulled from the WDI.

Published with the package.

https://datatopics.worldbank.org/world-development-indicators/

The Semantic Scholar Open Research Corpus

Filename: results_completed_updated_20231003.fst Lo, Kyle, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel Weld. 2020. “S2ORC: The Semantic Scholar Open Research Corpus.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 4969–83. Online: Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.447. This file contains article metadata for the 1 million articles classified by the machine learning algorithm. The file is too large to host on github, so it is hosted on S3 from AWS. The file is in fst format, which is a compressed format that is faster to read than csv.

Publicly hosted on S3 from AWS at "s3://data-use-academia/03_output/results_completed_updated_20231003.fst". It can also be downloaded from https://datacatalog.worldbank.org/int/search/dataset/0065200/data-use-in-academia-dataset by clicking on "data_use_dataset_fst"

https://datacatalog.worldbank.org/int/search/dataset/0065200/data-use-in-academia-dataset

Statistical Performance Indicators

Filename: SPI_index.csv Statistical Performance Indicators (Dang et al. 2023, Cameron et al. 2021)

Published with the package.

https://github.com/worldbank/SPI

Amazon Mturk

Filename: test_doc_wcounts.csv test_doc_wcounts.csv was produced by randomly selecting from the set of articles that were classified by human raters using Amazon Mturk. This subset of the articles that were classified by Mturk workers were used for testing the predictions from our machine learning model to classify data use of articles.

Published with the package.

Amazon Mturk

Filename: train_doc_wcounts.csv train_doc_wcounts.csv was a randomly selected set of articles that were classified by Amazon Mturk workers used for training our machine learning model. These articles were used to build our machine-learning model. The test and training set do not overlap.

Published with the package.

Das, Do, Shaines, and Srikant (2013) and Porteous (2022)

Filename: das_porteous_compare.csv das_porteous_compare.csv was a dataset our team prepared by reading in the articles produced per country from two papers: Das, Do, Shaines, and Srikant (2013) and Porteous (2022). Columns B-D of the spreadsheet were populated from Table A.3 of Das et al. (2013). Columns E-H were populated from Table 1 from Porteous (2022). Das et al. (2013) can be accessed at https://www.sciencedirect.com/science/article/pii/S0304387813001090#s0035 Porteous (2022) can be accessed at https://onlinelibrary.wiley.com/doi/epdf/10.1111/obes.12510

Published with the paper