Reproducibility package for Scaling Up Classroom Observation With Large Language Models: A Reliability Study Of The World Bank's Teach Framework In Peru

Classroom observation is widely used to measure teaching practices but is costly at scale, limiting its use as a formative tool for teacher improvement in low- and middle-income countries (LMICs). We test whether a large language model (LLM) can agree with a certified human observer as closely as the World Bank's Teach reliability protocol requires of trained raters. Across 199 Peruvian primary-classroom transcripts, we evaluate eight LLM configurations (Gemini 2.0 and 2.5 Flash) varying rubric quality, prompt format, and reasoning depth, and apply the Teach certification rule to the LLM-observer agreement via Monte Carlo simulation. Our best configuration meets the within-one-point certification criterion against the certified observer in 92.9% of simulated exams (95% CI [91.3, 94.5]), with element-level agreement of 81-99% across Teach elements—in the range trained human raters reach, though our gold standard is a single certified field observer rather than the program's master coders. A regression that isolates each design lever shows that rubric content and distributional priors are the two dominant drivers of agreement, while chain-of-thought reasoning and snapshot tagging are not; a minimal-rubric baseline almost never clears the criterion. A reliable LLM scorer can help ministries deliver formative feedback within teacher-support programs more frequently and at lower marginal cost.