Search for EdWorkingPapers here by author, title, or keywords.
Program and policy effects
Many districts and states have begun implementing incentives to attract high-performing teachers to low-performing schools. Previous research has found that these incentives are effective. However, effects on the schools and students these teachers leave behind has not been examined. This study focuses on the general equilibrium effects of recruiting effective teachers to Tennessee’s Innovation Zone (iZone) schools, one of the most successful turnaround initiatives in the nation (Zimmer, Henry, & Kho, 2017). While there is some variation in the effects of losing these teachers, we find they range between -0.04 and -0.12 standard deviations in student test score gains. However, an estimate including both these negative effects and the positive effects in iZone schools yields overall net positive effects.
Starting in 2011, Boston Public Schools (BPS) implemented major reforms to its teacher evaluation system with a focus on promoting teacher development. We administered independent district-wide surveys in 2014 and 2015 to capture BPS teachers’ perceptions of the evaluation feedback they receive. Teachers generally reported that evaluators were fair and accurate, but that they struggled to provide high-quality feedback. We conduct a randomized controlled trial to evaluate the district’s efforts to improve this feedback through an intensive training program for evaluators. We find little evidence the program affected evaluators’ feedback, teacher retention, or student achievement. Our results suggest that improving the quality of evaluation feedback may require more fundamental changes to the design and implementation of teacher evaluation systems.
Effect sizes in the Cohen’s d family are often used in education to compare estimates across studies, measures, and sample sizes. For example, effect sizes are used to compare gains in achievement students make over time, either in pre- and post-treatment studies or in the absence of intervention, such as when estimating achievement gaps. However, despite extensive research dating back to the paired t-test literature showing that such growth effect sizes should account for within-person correlations of scores over time, such achievement gains are often standardized relative to the standard deviation from a single timepoint or two timepoints pooled. Such a tendency likely occurs in part because there are not many large datasets from which a distribution of student- or school-level gains can be derived. In this study, we present a novel model for estimating student growth in conjunction with a national dataset to show that effect size estimates for student and school growth are often quite different when standardized relative to a distribution of gains rather than static achievement. In particular, we provide nationally representative empirical benchmarks for student achievement and gains, including for male-female gaps in those gains, and examine the sensitivity of those effect sizes to how they are standardized. Our results suggest that effect sizes scaled relative to a distribution of gains are less likely to understate the effects of interventions over time, and that resultant effect sizes often more closely match the estimand of interest for most practice, policy, and evaluation questions.
Despite broad public interest in Veterans' education, there is relatively little evidence documenting the postsecondary trajectories of military service members after they return to civilian life. In the current report we investigate how U.S. Army service member college enrollment and progression trends compare to a similar population of civilians, using Army administrative personnel data merged with administrative records from the National Student Clearinghouse and the Educational Longitudinal Study (ELS) of 2002. Civilians were nearly three times as likely to enroll in college within one year of high school graduation (or one year of separation). Civilians were also much more likely to earn a bachelor’s degree within the period of study than either of the Army samples. While members of minority race/ethnicity groups in both military samples enroll at higher rates than their white counterparts, racial/ethnic minorities do not graduate at higher rates than their white counterparts. We discuss policy implications of our analyses in the final section of our paper.
Field trips to see theater performances are a long-standing educational practice, however, there is little systematic evidence demonstrating educational benefits. This article describes the results of five random assignment experiments spanning two years where school groups were assigned by lottery to attend a live theater performance, or for some groups, watch a movie-version of the same story. We find significant educational benefits from seeing live theater, including higher levels of tolerance, social perspective taking, and stronger command of the plot and vocabulary of those plays. Students randomly assigned to watch a movie did not experience these benefits. Our findings also suggest that theater field trips may cultivate the desire among students to frequent the theater in the future.
Effective teacher hiring is fundamental to improving schools and yet few studies investigate this process. In this exploratory study of six successful, high-poverty schools (three charter, three district) in one Massachusetts city, we analyze the policy contexts that influenced hiring and examine the schools’ hiring practices. Through interviews with 142 teachers and administrators, we learned that, despite significant differences, these schools’ approaches were strikingly similar. Each used a two-way, information-rich hiring process that provided schools and candidates with opportunities to exchange information and assess one another before making an offer or signing a contract. Participants viewed their investment in hiring as an essential part of their school’s success. Based on our findings, we provide recommendations for policymakers, practitioners, and researchers.
Recent substantial increases in high school graduation rates have been linked anecdotally to the expansion of online course-taking for credit recovery. Online course-taking that supports high school completion could open new opportunities for postsecondary education pursuits. Alternatively, poorer quality online instruction could diminish student engagement and learning and discourage persistence toward graduation and further education. Using fixed-effect models and inverse probability weighting with regression adjustment with data from an eight-year longitudinal study of online course-taking in high schools, we find positive associations between online course-taking in high school and credits earned, high school graduation, and college enrollment. Our results leave open the question of whether online course-taking supports learning that will lead to longer-term postsecondary education and labor market success.
A growing body of research evaluates the effects of turnaround on chronically low-performing schools. We extend this research to formally test factors that either mediate or suppress the effects of two turnaround initiatives in Tennessee: the Achievement School District (ASD) and local Innovation Zones (iZones). Using difference-in-differences models within a mediational framework, we find that hiring highly effective teachers and employing effective principals partially explain positive effects of iZone interventions. In the ASD, high levels of teacher turnover suppress potential positive effects after the first year. In iZone schools, several factors suppress even larger positive effects: hiring more novice teachers; hiring more principals with less experience; and high levels of student chronic absenteeism and student in-migration.
Social programs and mandates are usually studied in isolation even though they often interact closely with each other. Given the immense recent changes to health insurance systems, there is much potential for spillover effects to other systems in which health plays a large role. In this study, we examine how health insurance interacts with education, specifically the education of students with disabilities. We present the first analysis in the literature of how a mandate for health insurers to cover therapy for Autism Spectrum Disorder (ASD) up to age 18 affects educational services received by, and test scores of, students with ASD. A key aspect of the mandate is that children covered by Medicaid aged out of benefits quickly (by age 6), leaving them with a far weaker benefit than children covered by private insurance. Since we do not observe insurance status directly, we proxy for private insurance coverage using ineligibility for free/reduced-price lunch (FRPL) and estimate impacts on identification with ASD, special education services, and achievement through a series of difference-in-differences and triple difference models. We find little evidence of an overall shift in ASD identification, but we do find substantial crowd-out of special education services for students with ASD from the mandate. The stronger mandate led to increased mainstreaming of students in general education classrooms and a reduction in special education support services like teacher consultants. Girls in particular are more likely to be mainstreamed. There is little evidence of changes in achievement, which supports our interpretation of the service reductions as crowd-out.
The College Board sought to reduce barriers in the college application process by minimizing information aggregation costs, encouraging a broad application portfolio, and providing an impetus to start the search process. Some students were offered additional encouragements, such as text message reminders or college application fee waivers. In a randomized control trial with 785,000 low- and middle-income students in the top 50% of the PSAT and SAT distributions, we find no changes in college enrollment patterns, with the exception of a 0.02σ increase in college quality measures for African-American and Hispanic students.