Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

James S. Kim, Patrick Rich, Ethan Scherer.

To address the challenge of improving third grade reading comprehension, we developed and evaluated the long-term effects of a sustained content literacy intervention called the Model of Reading Engagement (MORE), which emphasizes building domain and topic knowledge schemas from Grade 1 to Grade 3. The MORE intervention emphasizes thematic lessons that provide an intellectual framework for helping students connect new learning to a general schema (e.g., how scientists study past events, how systems function properly). Over three years, the treatment group students participated in (a) spring Grade 1 thematic content literacy lessons in science and social studies, (b) fall to spring Grade 2 thematic content literacy lessons in science, (c) remote Grade 3 thematic content literacy lessons in science, and (d) wide reading of thematically related informational texts in the summer months following Grade 1 and Grade 2. During the third grade school year (SY 2020-21), the COVID-19 pandemic required remote schooling to be in place from fall to spring and the Grade 3 MORE was provided to both treatment and control students. Accordingly, we examine long-term effects on third graders’ outcomes comparing a treatment group that received the Grade 1, Grade 2, and Grade 3 MORE treatment to a control condition that received the Grade 3 MORE treatment. Intent-to-treat estimates show that the students randomly assigned to the treatment condition outperformed control students in reading comprehension (ES = 0.11) and mathematics (ES = 0.14) on third grade state standardized assessments. Subgroup analyses also revealed positive impacts for student living in low- to moderate-socioeconomic status neighborhoods on both reading comprehension (ES = .13) and mathematics (ES = .20). Findings indicate that a sustained content literacy intervention may be a scalable approach for accelerating and equalizing third-graders’ reading comprehension and math outcomes.

More →


David Blazar, Melinda Adnot, Xinyi Zhong.

We examine heterogenous responses to job-embedded performance incentives along two dimensions theorized to drive motivation: (i) expectations of success when faced with easier versus more difficult tasks, and (ii) ones’ social identity as part of a marginalized group (in our case, race). Compared to the largely lab-based literature on this topic, we leverage data from a fully implemented teacher evaluation system from the District of Columbia Public Schools over a seven-year period (2009-10 through 2015-16). Our regression discontinuity estimates reveal not only that task difficulty and social/racial identity drive much of the incentive effects, but also that there is a strong interaction between the two. Low-performing teachers threatened with dismissal improved much more on tasks with low difficulty and high expectations of success, relative to more difficult tasks (roughly 0.3 SD versus 0.15 SD). These trends were particularly pronounced for Black teachers, who experienced fewer successes than White teachers in the evaluation system generally. We also find that high-performing Black teachers were less responsive than White teachers to an incentive to increase their base pay. At the same time, the responses of Black teachers to the salary incentive were malleable and tracked closely with district-led redesign efforts that aimed to ensure greater equity in terms of teachers most likely to reap the benefits of this incentive.

More →


Devon L. Graves.

Every year millions of students seeking access to federal financial aid complete the Free Application for Federal Student Aid (FAFSA) application which grants an estimated $234 billion in federal aid in the 2020-21 academic year. Upon receiving students’ FAFSA, the U.S. Department of Education selects some students for income verification, a process in which educational institutions check the accuracy of the information students filled out on the FAFSA. I conducted semi-structured interviews with 17 Latinx community college students to identify barriers in the verification process. Using Critical Race Theory, I contend the verification process reflects and upholds institutional racism within the financial aid process through three barriers. Latinx students experience concern and confusion upon receiving notification of verification selection, difficulty locating requested documentation and acquiring parents’ signature, and undergo a lengthy review of their verification forms which delays receipt of their financial aid.

More →


Ariana Audisio, Rebecca Taylor-Perryman, Tim Tasker, Matthew P. Steinberg.

Teachers are the most important school-specific factor in student learning. Yet, little evidence exists linking teacher professional learning programs and the various strategies or components that comprise them to student achievement. In this paper, we examine a teacher fellowship model for professional learning designed and implemented by Leading Educators, a national nonprofit organization that aims to bridge research and practice to improve instructional quality and accelerate learning across school systems. During the 2015-16 and 2016-17 school years, Leading Educators conducted its fellowship program for teachers and school leaders to provide educators ongoing, collaborative, job-embedded professional development and to improve student achievement. Relying on quasi-experimental methods, we find that a school’s participation in the fellowship model increased student proficiency rates in math and English language arts on state achievement exams. Further, student achievement benefitted from a more sustained duration of teacher participation in the fellowship model, and the impact on student achievement varied depending on the share of a school’s teachers who participated in the fellowship model and the extent to which teachers independently selected into the fellowship model or were appointed to participate by school leaders. Taken together, findings from this paper should inform professional learning organizations, schools, and policymakers on the design, implementation and impact of teacher professional learning.

More →


Dominique J. Baker, Karly S. Ford, Samantha Viano, Marc P. Johnston-Guerrero.

How scholars name different racial groups has powerful salience for understanding what researchers study. We explored how education researchers used racial terminology in recently published, high-profile, peer-reviewed studies. Our sample included all original empirical studies published in the non-review AERA journals from 2009 to 2019. We found two-thirds of articles used at least one racial category term, with an increase from about half to almost three-quarters of published studies between 2009 and 2019. Other trends include the increasing popularity of the term Black, the emergence of gender-expansive terms such as Latinx, the popularity of the term Hispanic in quantitative studies, and the paucity of studies with terms connoting missing race data or including terms describing Indigenous and multiracial peoples.

More →


Emily Rauscher, Yifan Shen.

The equity-efficiency tradeoff and cumulative return theories predict larger returns to school spending in areas with higher previous investment in children. Equity – not efficiency – is therefore used to justify progressive school funding: spending more in communities with fewer financial resources. Yet it remains unclear how returns to school spending vary across areas by previous investment. Using county-level panel data 2009-2018 from the Stanford Education Data Archive, the F-33 finance survey, and National Vital Statistics, we estimate achievement returns to school spending and test whether returns vary between counties with low and high levels of initial human capital (measured as birth weight), child poverty, and previous spending. Spending returns are higher among counties with low previous investment (counties that also have a high percent of Black students). Evidence of diminishing returns by previous investment documents another way that schools increase equality and establishes another argument for progressive school funding: efficiency.

More →


Todd Pugatch, Elizabeth Schroeder, Nicholas Wilson.

We design a commitment contract for college students, "Study More Tomorrow," and conduct a randomized control trial testing a model of its demand. The contract commits students to attend peer tutoring if their midterm grade falls below a pre-specified threshold. The contract carries a financial penalty for noncompliance, in contrast to other commitment devices for studying tested in the literature. We find demand for the contract, with take-up of 10% among students randomly assigned a contract offer. Contract demand is not higher among students randomly assigned to a lower contract price, plausibly because a lower contract price also means a lower commitment benefit of the contract. Students with the highest perceived utility for peer tutoring have greater demand for commitment, consistent with our model. Contrary to the model's predictions, we fail to find evidence of increased demand among present-biased students or among those with higher self-reported tendency to procrastinate. Our results show that college students are willing to pay for study commitment devices. The sources of this demand do not align fully with behavioral theories, however.

More →


Leigh Wedenoja, John Papay, Matthew A. Kraft.

We examine the dynamic nature of student-teacher match quality by studying the effect of having a teacher for more than one year. Using data from Tennessee and panel methods, we find that having a repeat teacher improves achievement and decreases absences, truancy, and suspensions. These results are robust to a range of tests for student and teacher sorting. High-achieving students benefit most academically and boys of color benefit most behaviorally. Effects increase with the share of repeat students in a class suggesting that classroom assignment policies intended to promote sustained student-teacher relationships such as looping may have even larger benefits.

More →


Todd Pugatch, Paul Thompson.

Can public university honors programs deliver the benefits of selective undergraduate education within otherwise nonselective institutions? We evaluate the impact of admission to the Honors College at Oregon State University, a large nonselective public university. Admission to the Honors College depends heavily on a numerical application score. Nonlinearities in admissions probabilities as a function of this score allow us to compare applicants with similar scores, but different admissions outcomes, via a fuzzy regression kink design. The first stage is strong, with takeup of Honors College programming closely following nonlinearities in admissions probabilities. To estimate the causal effect of Honors College admission on human capital formation, we use these nonlinearities in the admissions function as instruments, combined with course-section fixed effects to account for strategic course selection. Honors College admission increases course grades by 0.10 grade points on the 0-4 scale, or 0.14 standard deviations. Effects are concentrated at the top of the course grade distribution. Previous exposure to Honors sections of courses in the same subject is a leading potential channel for increased grades. However, course grades of first-generation students decrease in response to Honors admission, driven by low performance in natural science courses. Results suggest that selective Honors programs can accelerate skill acquisition for high-achieving students at public universities, but not all students benefit from Honors admission.

More →


Daniel Rodriguez-Segura, Beth E. Schueler.

A significant share of education and development research uses data collected by workers called “enumerators.” It is well-documented that “enumerator effects”—or inconsistent practices between the individual people who administer measurement tools— can be a key source of error in survey data collection. However, it is less understood whether this is a problem for academic assessments or performance tasks. We leverage a remote phone-based mathematics assessment of primary school students and survey of their parents in Kenya. Enumerators were randomized to students to study the presence of enumerator effects. We find that both the academic assessment and survey was prone to enumerator effects and use simulation to show that these effects were large enough to lead to spurious results at a troubling rate in the context of impact evaluation. We therefore recommend assessment administrators randomize enumerators at the student level and focus on training enumerators to minimize bias.

More →