Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Program and policy effects

Eric S. Taylor.

When an employee expects repeated evaluation and performance incentives over time, the potential future rewards create an incentive to invest in building relevant skills. Because new skills benefit job performance, the effects of an evaluation program can persist after the rewards end or even anticipate the start of rewards. I test for persistence and anticipation effects, along with more conventional predictions, using a quasi-experiment in Tennessee schools. Performance improves with new evaluation measures, but gains are larger when the teacher expects future rewards linked to future scores. Performance rises further when incentives start and remains higher even after incentives end.

More →


Darryl V. Hill, Rodney P. Hughes, Matthew A. Lenard, David D. Liebowitz, Lindsay C. Page.

Policy makers periodically consider using student assignment policies to improve educational outcomes by altering the socio-economic and academic skill composition of schools. We exploit the quasi-random reassignment of students across schools in the Wake County Public School System to estimate the academic and behavioral effects of being reassigned to a different school and, separately, of shifts in peer characteristics. We rule out all but substantively small effects of transitioning to a different school as a result of reassignment on test scores, course grades and chronic absenteeism. In contrast, increasing the achievement levels of students' peers improves students' math and ELA test scores but harms their ELA course grades. Test score benefits accrue primarily to students from higher-income families, though students with lower family income or lower prior performance still benefit. Our results suggest that student assignment policies that relocate students to avoid the over-concentration of lower-achieving students or those from lower-income families can accomplish equity goals (despite important caveats), although these reassignments may reduce achievement for students from higher-income backgrounds.

More →


Danielle Sanderson Edwards, Matthew A. Kraft, Alvin Christian, Christopher A. Candelaria.

We develop a unifying conceptual framework for understanding and predicting teacher shortages at the state, region, district, and school levels. We then generate and test hypotheses about geographic, grade level, and subject variation in teacher shortages using data on teaching vacancies in Tennessee during the fall of 2019. We find that teacher staffing challenges are highly localized, causing shortages and surpluses to coexist. Aggregate descriptions of staffing challenges mask considerable variation between schools and subjects within districts. Schools with fewer local early-career teachers, smaller district salary increases, worse working conditions, and higher historical attrition rates have higher vacancy rates. Our findings illustrate why viewpoints about, and solutions to, shortages depend critically on whether one takes an aggregate or local perspective.

More →


Matthew A. Kraft, Melissa Arnold Lyon.

We examine the state of the U.S. K-12 teaching profession over the last half century by compiling nationally representative time-series data on four interrelated constructs: professional prestige, interest among students, preparation for entry, and job satisfaction. We find a consistent and dynamic pattern across every measure: a rapid decline in the 1970s, a swift rise in the 1980s, relative stability for two decades, and a sustained drop beginning around 2010. The current state of the teaching profession is at or near its lowest levels in 50 years. We identify and explore a range of factors that might explain these historical patterns including education funding, teacher pay, outside opportunities, unionism, barriers to entry, working conditions, accountability, autonomy, and school shootings.  

More →


Matthew A. Kraft, Megan Conklin, Grace Falken.

We examine the labor supply decisions of substitute teachers – a large, on-demand market with broad shortages and inequitable supply. In 2018, Chicago Public Schools implemented a targeted bonus program designed to reduce unfilled teacher absences in largely segregated Black schools with historically low substitute coverage rates. Using a regression discontinuity design, we find that incentive pay substantially improved coverage equity and raised student achievement. Changes in labor supply were concentrated among Black and Hispanic substitutes from nearby neighborhoods with experience in incentive schools. Wage elasticity estimates suggest incentives would need to be 50% of daily wages to close fill-rate gaps.

More →


Seth B. Hunter, Matthew P. Steinberg.

Districts nationwide have revised their educator evaluation systems, increasing the frequency with which administrators observe and evaluate teacher instruction. Yet, limited insight exists on the role of evaluator feedback for instructional improvement. Relying on unique observation-level data, we examine the alignment between evaluator and teacher assessments of teacher instruction and the potential consequences for teacher productivity and mobility. We show that teachers and evaluators typically rate teacher performance similarly during classroom observations, but with significant variability in teacher-evaluator ratings. While teacher performance improves across multiple classroom observations, evaluator ratings likely overstate productivity improvements among the lowest-performing teachers. Evaluators, but not teachers, systematically rate teacher performance lower in classrooms serving higher concentrations of economically disadvantaged students. And while teacher performance improves when evaluators provide more critical feedback about teacher instruction, teachers receiving critical feedback may seek alternative teaching assignments in schools with less critical evaluation settings. We discuss the implications of these findings for the design, implementation and impact of educator evaluation systems.

More →


Carly D. Robinson, Biraj Bisht, Susanna Loeb.

Billions of dollars are invested in opt-in, educational resources to accelerate students’ learning. Although advertised to support struggling, marginalized students, there is no guarantee these students will opt in. We report results from a school system’s implementation of on-demand tutoring. The take up was low. At baseline, only 19% of students ever accessed the platform, and struggling students were far less likely to opt in than their more engaged and higher achieving peers. We conducted a randomized controlled trial (N=4,763) testing behaviorally-informed approaches to increase take-up. Communications to parents and students together increase the likelihood students access tutoring by 46%, which led to a four-percentage point decrease in course failures. Nonetheless, take-up remained low, showing concerns that opt-in resources can increase—instead of reduce—inequality are valid. Without targeted investments, opt-in educational resources are unlikely to reach many students who could benefit.

More →


Carycruz Bueno, Lindsay C. Page, Jonathan Smith.

We investigate whether and how Achieve Atlanta’s college scholarship and associated services impact college enrollment, persistence, and graduation among Atlanta Public School graduates experiencing low household income.  Qualifying for the scholarship of up to $5,000/year does not meaningfully change college enrollment among those near the high school GPA eligibility thresholds. However, scholarship receipt does have large and statistically significant effects on early college persistence (i.e., 14%) that continue through BA degree completion within four years (22%).  We discuss how the criteria of place-based programs that support economically disadvantaged students may influence results for different types of students.

More →


Juan Pablo Valenzuela, Eduardo A. Undurraga, Danilo Kuzmanic, Andrea Canales, Susana Claro, Fernanda Cortés.

The disruption of in-person schooling during the Covid-19 pandemic has affected students’ learning, development, and well-being. Students in Latin America and the Caribbean have been hit particularly hard because schools in the region have stayed closed for longer than anywhere else, with long-term expected adverse consequences. Little is known about which factors are associated with the slow in-person return to school in the region and how these factors have had differential effects based on students’ socio-economic status. Combining a longitudinal national survey of the Chilean school system and administrative datasets, we study the supply and demand factors associated with students’ resuming in-person instruction and the socio-economic gaps in school reopening in Chile in 2021. We defined socio-economic status based on parents’ education and household income. Our results show that in-person learning in 2021 was limited mainly by supply factors (i.e., sanitary, administrative, and infrastructure restrictions). However, once the supply restrictions decreased, many low-income students and their families did not resume in-person instruction. We found vast inequalities in face-to-face instruction by school’s socio-economic characteristics. On average, schools in the highest 10% of the socio-economic distribution had three times higher attendance rates than the remaining 90%. We found no significant differences between schools in the lowest 90% of the distribution. After exceptionally long school closures, most school authorities, students, and their families did not return to in-person instruction, particularly those of low socio-economic status. These inequalities in in-person instruction will expand existing disparities in students’ learning and educational opportunities.

More →


Jilli Jung, Maithreyi Gopalan.

Youth voter turnout remains stubbornly low and unresponsive to civic education. Rigorous evaluations of the adoption of civic tests for high school graduation by some states on youth voter turnout remain limited. We estimate the impact of a recent, state-mandated civics test policy—the Civics Education Initiative (CEI)—on youth voter turnout by exploiting spatial and temporal variation in the adoption of CEI across states. Using nationally-representative data from the 1996-2020 Current Population Survey and a Difference-in-Differences analysis, we find that CEI does not significantly affect youth voter turnout. Our null results, largely insensitive to a variety of alternative specifications and robustness checks, provide evidence regarding the lack of efficacy of civic test policies when it comes to youth voter participation.

More →