Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Program and policy effects

Kaitlyn O'Hagan, Leanna Stiefel, Amy Ellen Schwartz.

Middle school transitions are increasingly required, despite documented negative effects on general education students (GENs). We explore if and how the move to middle school differentially affects students with disabilities (SWDs), a large and low-performing group of students. Using an instrumental variables strategy and NYC data on nine cohorts of students, we find the middle school transition causes a 0.29 standard deviation decline in SWD math performance, a 0.16 standard deviation decline in ELA performance, and a one percentage point increase in grade retention. However, after accounting for potential mediators (e.g. peer cohort stability) effects are similar for SWDs and GENs, suggesting the need to ease the middle school transition for all students.

More →


Sara White, Leiah Groom-Thomas, Susanna Loeb.

Tutoring has emerged as an especially promising strategy for supporting students academically. This study synthesizes 33 articles on the implementation of tutoring, defined as one-to-one or small-group instruction in which a human tutor supports students grades K-12 in an academic subject, to better understand the facilitators and barriers to program success. We find that policies influenced tutoring implementation through the allocation of federal funding and stipulation of program design. Tutoring program launch has often been facilitated by strategic relationships between schools and external tutoring providers and strengthened by transparent assessments of program quality and effectiveness. Successful implementation hinged on the support of school leaders with the power to direct school funding, space, and time. Tutoring setting and schedule, recruitment and training, and curriculum influenced whether students are able to access quality tutoring and instruction. Ultimately, evidence suggests that tutoring was most meaningful when tutors fostered positive student-tutor relationships which they drew upon to target instruction toward students’ strengths and needs.

More →


Erica Harbatkin, Jason Burns, Samantha Cullum.

School climate is critical to school effectiveness, but there is limited large-scale data available to examine the magnitude and nature of the relationship between school climate and school improvement. Drawing on statewide administrative data linked with unique teacher survey data in Michigan, we examine whether school climate appeared to play a role in the effects of a state-level school turnaround intervention. Using comparative interrupted time series models and descriptive mediation analysis, we find that students in schools with more positive school climate appeared to fare better than their peers in schools with less positive climate. Certain elements of climate—relational trust and school leadership—also mediated the effect of turnaround on student achievement. Our findings have implications for school improvement planning, for the design of evaluations of school turnaround initiatives, and for data collection by states aiming to improve their lowest performing schools.

More →


David Grissmer, Thomas White, Richard Buddin, Mark Berends, Daniel Willingham, Jamie DeCoster, Chelsea Duran, Chris Hulleman, William Murrah, Tanya Evans.

The Core Knowledge curriculum is a K-8 curriculum focused on building students General Knowledge about the world they live in that is hypothesized to increase reading comprehension and Reading/English-LA achievement. This study utilizes an experimental design to evaluate the long term effects of attending Charter schools teaching the Core Knowledge curriculum. Fourteen oversubscribed kindergarten lotteries for enrollment in nine Core Knowledge Charter schools using the curriculum had 2310 students applying from parents in predominately middle/high income school districts. State achievement data was collected at 3rd- 6th grade in Reading/English-LA and Mathematics and at 5th Grade in Science. A new methodology addresses two previously undiscovered sources of bias inherent in kindergarten lotteries that include middle/high income families. The unbiased confirmatory Reading-English-LA results show statistically significant ITT (0.241***) and TOT (0.473***) effects for 3rd-6th grade achievement with statistically significant ITT and TOT effects at each grade. Exploratory analyses also showed significant ITT (0.15*) and TOT (0.300*) unbiased effects at 5th grade in Science. A CK-Charter school in a low income school district also had statistically significant, moderate to large unbiased ITT and TOT effects in English Language Arts (ITT= 0.944**; TOT = 1.299**), Mathematics (ITT= 0.735*; TOT = 0.997*) and positive, but insignificant Science effects (ITT= 0.468; TOT = 0.622) that eliminated achievement gaps in all subjects.

More →


Sade Bonilla, Alexander Thim.

In this study, we examine an at-scale effort to encourage the formation of career pathways in California, with the goal of estimating the initiative’s causal effects on community college enrollment. We leverage a discontinuous assignment rule used to award grant funds to obtain credibly causal estimates of an ambitious $500 million effort to expand and establish career and technical education pathways between K-12 and community colleges. The competitive grant application process used a standardized rubric, and those receiving a score above a predetermined threshold were awarded funding (i.e., treatment group) while those just below received no funding (i.e., control group), allowing for a regression discontinuity (RD) design. We found that successful grantees did not experience overall enrollment increases in postsecondary partnerships; however, there were enrollment increases of 13.5 percent to 14.8 percent in CTE health sector courses, the program targeted the most for expansion. Manufacturing, and information communication technology, the other programs with the most expanded offerings, experienced no increases in postsecondary enrollment. The enrollment increases for the health sector were concentrated amongst female students in line with earlier findings by Bonilla (2020) documenting reductions in high school dropout rates for female students. These findings suggest that partnerships between K-12 schools and postsecondary institutions may be a viable avenue for increasing alignment between enrollment and high- growth sectors.

More →


Brian Holzman, Irina Chukhray, Courtney Thrash.

There is a growing debate in social science and education policy research on how to improve college access for high-performing students from low-income or first-generation backgrounds. While some studies suggest that providing information to students impacts college access, other studies do not and suggest that students may need more support in the college search and choice processes. Using a regression discontinuity research design with a layered randomized controlled trial, this study examines how information and personal assistance impact SAT scores, college application behaviors, and college enrollment decisions among low-income and first-generation high school students in a large urban school district. The results show that an intensive, multi-year college access program has large, positive effects on applying to a selective college, the number of applications submitted to selective colleges, and enrollment in a selective college. In contrast, a low-touch, general information packet intervention shows null effects on these outcomes. Implications for future nudge interventions and scaling up social capital interventions are discussed.

More →


Clemence Darriet, Lucrecia Santibanez.

Purpose. Bilingual programs in the United States, particularly two-way dual language immersion (TWDL) programs, have been implemented since the 1960s to support the education of English Learner-classified (EL-classified) and language minoritized students. Over the past decade, TWDL programs have grown significantly across the United States. This study examines TWDL program growth in Los Angeles Unified School District, exploring the relationships between program expansion and neighborhood change, enrollment declines, and school choice. These factors have been linked to decreased access to these programs for language minoritized students. Research Methods/Approach. We descriptively examine the neighborhood characteristics of TWDL schools over a 22-year period using publicly available school, census, and housing data, and investigate the relationship between these factors and TWDL emergence. Findings. We find that of the three factors we explored, enrollment change (specifically declining enrollment) and the existence of nearby charter schools are two factors most likely to be associated with TWDL program emergence. We find little evidence that TWDL are primarily emerging in gentrifying contexts. Implications. This study helps us understand general, decade-long trends of TWDL program expansion and dispersion in a district undergoing many of the phenomena described in the literature on this topic.

More →


Jesse Bruhn, Christopher Campos, Eric Chyn.

We study the distributional effects of remote learning. Our approach combines newly collected data on parental preferences with administrative data from Los Angeles. The preference data allow us to account for selection into remote learning while also studying selection patterns and treatment effect heterogeneity. We find a negative average effect of remote learning on reading (–0.14σ) and math (–0.17σ). Notably, we find evidence of positive learning effects for children whose parents have the strongest demand for remote learning. Our results suggest an important subset of students who currently sort into post-pandemic remote learning benefit from expanded choice.

More →


David Figlio, Krzysztof Karbownik, Umut Özek.

Public policies often target individuals but within-family externalities of such interventions are understudied. Using a regression discontinuity design, we document how a third grade retention policy affects both the target children and their younger siblings. The policy improves test scores of both children while the spillover is up to 30% of the target child effect size. The effects are particularly pronounced in families where one of the children is disabled, for boys, and in immigrant families. Candidate mechanisms include improved classroom inputs and parental school choice.

More →


Gautam Anand, Aishwarya Atluri, Lee Crawfurd, Todd Pugatch, Ketki Sheth.

Improving school quality in low and middle income countries (LMICs) is a global priority. One way to improve quality may be to improve the management skills of school leaders. In this systematic review, we analyze the impact of interventions targeting school leaders' management practices on student learning. We begin by describing the characteristics and responsibilities of school leaders using data from large, multi-country surveys. Second, we review the literature and conduct a meta-analysis of the causal effect of school management interventions on student learning, using 39 estimates from 20 evaluations. We estimate a statistically significant improvement in student learning of 0.04 standard deviations. We show that effect sizes are not related to program scale or intensity. We complement the meta-analysis by identifying common limitations to program effectiveness through a qualitative assessment of the studies included in our review. We find three main factors which mitigate program effectiveness: 1) low take-up; 2) lack of incentives or structure for implementation of recommendations; and 3) the lengthy causal chain linking management practices to student learning. Finally, to assess external validity of our review, we survey practitioners to compare characteristics between evaluated and commonly implemented programs. Our findings suggest that future work should focus on generating evidence on the marginal effect of common design elements in these interventions, including factors that promote school leader engagement and accountability.

More →