Assessing the "Rothstein Falsification Test": Does It Really Show Teacher Value-Added Models Are Biased?

In an influential paper, Jesse Rothstein (2010) shows that standard value-added models (VAMs) suggest implausible and large future teacher effects on past student achievement. This is the basis of a falsification test that appears to indicate bias in typical VAM estimates of teacher contributions to student learning on standardized tests. We find that the "Rothstein falsification test" generates large and implausible future teacher effects even when students are randomly assigned to classrooms. Importantly, we also show that although the test can determine when students are tracked to teachers, this tracking result does not necessarily imply bias in estimates of teacher effectiveness. The primary reason that the test does not imply bias is that tracking could be based on lagged student achievement, which is usually accounted for in VAMs. Specifically, we show theoretically and through simulations that the test proposed in Rothstein will often falsify VAMs that are unbiased and fail to falsify VAMs that are biased.