Erica Harbatkin

The Every Student Succeeds Act (ESSA) began a new wave of school accountability under which states draw on multiple measures to assess school quality. States have options in terms of how to weight components in their school quality indices and how many years of data to use to determine school ratings. In this study, we simulate school ratings using eight years of administrative data in North Carolina to demonstrate how state decisions about school ratings and identification influence school ratings and the list of schools identified for improvement. We then evaluate these decisions against a framework that considers the validity, stability, and equity of the ratings, underscoring the inherent tradeoffs that come with each. We show that while a system that weights proficiency more heavily than growth produces more stable school ratings, identifying schools based on multiple years of performance data instead of one more than offsets the loss of stability in shifting to a growth measure. We conclude with recommendations for state accountability systems under ESSA and for federal policymaking moving forward.

More →

Teacher turnover is a perennial concern that became more salient during the COVID-19 pandemic as teacher-reported intentions to leave teaching escalated. The extent to which these teacher reports may translate into actual turnover remains an open question—especially given the pandemic context. Using unique survey data from teachers in 35 districts in Michigan linked to statewide administrative data, we examine the extent to which teacher-reported intentions are predictive of actually leaving. We measure behavior one, two, and three years following reported intent. We find intent is a significant predictor of turnover and becomes increasingly predictive over time. We also find organizational commitment and school organizational conditions are important factors in teachers’ intent and, to a lesser degree, actual turnover behavior.

More →

Turnaround schools and districts that were charged with making rapid and dramatic improvements before the COVID-19 pandemic struck faced considerable challenges carrying out improvement efforts during pandemic schooling. Using survey and administrative data collected during the pandemic, we document some of the ways in which students and educators in Michigan’s turnaround schools and districts experienced the pandemic. We show that the communities in which turnaround schools are located were hardest hit by the pandemic and school and district operations were substantially disrupted. By extension, turnaround districts and especially the lowest performing schools in those districts that were targeted for school-level turnaround experienced high rates of student absenteeism, low student and parent engagement, and, ultimately, significantly smaller gains on math and reading benchmark assessments than in non-turnaround districts. Our findings have implications for policy as states amplify school and district turnaround efforts that were disrupted by the pandemic.

More →

School climate is critical to school effectiveness, but there is limited large-scale data available to examine the magnitude and nature of the relationship between school climate and school improvement. Drawing on statewide administrative data linked with unique teacher survey data in Michigan, we examine whether school climate appeared to play a role in the effects of a state-level school turnaround intervention. Using comparative interrupted time series models and descriptive mediation analysis, we find that students in schools with more positive school climate appeared to fare better than their peers in schools with less positive climate. Certain elements of climate—relational trust and school leadership—also mediated the effect of turnaround on student achievement. Our findings have implications for school improvement planning, for the design of evaluations of school turnaround initiatives, and for data collection by states aiming to improve their lowest performing schools.

More →

Starting in 2009, the U.S. public education system undertook a massive effort to institute new high-stakes teacher evaluation systems. We examine the effects of these reforms on student achievement and attainment at a national scale by exploiting the staggered timing of implementation across states. We find precisely estimated null effects, on average, that rule out impacts as small as 1.5 percent of a standard deviation for achievement and 1 percentage point for high school graduation and college enrollment. We also find little evidence of heterogeneous effects across an index measuring system design rigor, specific design features, and district characteristics.

More →

Test-based accountability pressures have been shown to result in transferring less effective teachers into untested early grades and more effective teachers to tested grades. In this paper, we evaluate whether a state initiative to turnaround its lowest performing schools reproduced a similar pattern of assigning teachers and unintended, negative effects on the outcomes of younger students in untested grades. Using a sharp regression discontinuity design, we find consistent evidence of increased chronic absenteeism and grade retention in the first year. Also, the findings suggest negative effects on early literacy and reading comprehension in the first year of the reform that rebounded somewhat in the second year. Schools labeled low performing reassigned low effectiveness teachers from tested grades into untested early grades, though these assignment practices were no more prevalent in reform than control schools. Our results suggest that accountability-driven school reform can yield negative consequences for younger students that may undermine the success and sustainability of school turnaround efforts.

More →

In contrast to prior federally mandated school reforms, the Every Student Succeeds Act (ESSA) allows states more discretion in reforming their lowest performing schools, removes requirements to disrupt the status quo, and does not allocate substantial additional funds. Using a regression discontinuity design, we evaluate a state turnaround initiative aligned with ESSA requirements. We find the effect on student test score growth was not significant in year one and -0.13 in year two. Also in year two, we find that teachers in turnaround schools were 22.5 percentage points more likely to turn over. Teacher turnover appears to have been voluntary rather than the result of strategic staffing decisions.

More →

One in five schools loses its principal each year. Despite the prevalence of principal turnover, little empirical research has examined its effects on school outcomes. Because principal turnover may occur in response to or contemporaneous with a downturn in student achievement, the effect of a turnover is confounded with unobserved school-level factors. We employ a novel identification strategy that blocks each potential source of endogeneity to isolate plausibly causal effects of within- and between-year principal turnover. Using eight years of North Carolina administrative data from 2009-2018, we find that principal turnover is associated with significant decreases in student achievement and increases in teacher turnover. These effects are similar whether the turnover occurs over the summer or during the school year.

More →