Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Susanna Loeb

Christopher Doss, Hans Fricke, Susanna Loeb, Justin B. Doromal.

This study assesses the effects of two text messaging programs for parents that aim to support the development of math skills in prekindergarten students. One program focuses purely on math, while the other takes an identical approach but focuses on a combination of math, literacy, and social-emotional skills. We find no evidence that the math-only program benefits children’s math development. However, the combination program shows greater promise, particularly for girls. Quantile regressions indicate that the effects are concentrated in the lower half of the outcome distribution. We discuss and provide evidence for various hypotheses that could explain these differences.

Highlights:

  • We test the effects of two text messaging curricula that leverage behavioral economics principles to help parents support the math development of prekindergarteners in the home.
  • We find that a program that cycles through literacy, mathematics, and social-emotional skills increases math achievement for girls, while a program focusing solely on mathematics has no effects.
  • Benefits for girls are concentrated on those with weaker performance on mathematics assessments.
  • We posit potential mechanisms based on the literature.

More →

Leiah Groom-Thomas, Monica Lee, Cate Smith Todd, Kathleen Lynch, Susanna Loeb, Scott McConnell, Lydia Carlis.

Many preschool agencies nationwide continue to experience closures and/or conversions to virtual or hybrid instruction due to the ongoing COVID-19 pandemic. Despite the importance of understanding young children’s learning and development during the COVID emergency, limited knowledge exists on adaptable practices of assessing young children during the pandemic. We detail practices used to assess learning in 336 Head Start children across four states during three different time periods in the 2020-21 school year, using adaptation of traditionally in-person assessments of early numeracy, early literacy, and executive functioning. In doing so, we distill early lessons for the field from the application of a novel, virtual assessment method with the early childhood population. The paper describes adaptations of assessment administration for virtual implementation and incorporation of feedback into continued virtual delivery of assessments. Applications and limitations in broader contexts are discussed.

More →

Kathleen Lynch, Monica Lee, Susanna Loeb.

The COVID-19 pandemic’s impact on preschool children’s school readiness skills remains understudied. This research investigates whether exposure to in-person (versus virtual) Head Start preschool predicted children’s early numeracy, literacy, and executive function outcomes during a pandemic-affected school year, using a novel virtual assessment methodology. Study children (N = 336; mean age = 51 months; 46% Hispanic; 36% Black Non-Hispanic; 52% female) experienced low in-person preschool exposure compared to national pre-pandemic norms. However, study children experienced gains during the pandemic-affected year of 0.08 SD in executive function, 0.34 SD in print knowledge, and 0.49-0.75 in early numeracy skills. For two of the three early numeracy domains measured, spring test score outcomes were stronger among children who attended more in-person preschool.

More →

Biraj Bisht, Zachary LeClair, Susanna Loeb, Min Sun.

Paraeducators perform multiple roles in U.S. classrooms, including among others preparing classroom activities, working with students individually and in small groups, supporting individualized programming for students with disabilities, managing classroom behavior, and engaging with parents and communities. Yet, little research provides insights into this key group of educators. This study combines an analysis of national administrative data to describe the paraeducator labor market with a systematic review of collective bargaining agreements and other job-defining documents in ten case-study districts. We find a large and expanding labor market of paraeducators, far more diverse along ethnic and racial lines than certified teachers but with far lower wages, fewer performance incentives, less professional development, and fewer opportunities for advancement within the profession.

More →

Carly D. Robinson, Susanna Loeb.

Research consistently demonstrates that tutoring interventions have substantial positive effects on student learning. As a result, tutoring has emerged as a promising strategy for addressing COVID-related learning loss and affording greater educational opportunities for students living in poverty. The effectiveness of tutoring programs, however, varies greatly, and these variations may drive differential gains in student learning. Therefore, determining the program characteristics that do and do not drive positive student outcomes will be key to providing guidance for policymakers and practitioners who want to implement high-impact tutoring at scale. Our goal is to highlight the programs, characteristics, and conditions that evidence suggests make for effective tutoring and to create an evidence-based framework for delivering and evaluating tutoring interventions. In addition, we identify promising questions for future research.

More →

Heather C. Hill, Zid Mancenido, Susanna Loeb.

Despite calls for more evidence regarding the effectiveness of teacher education practices, causal research in the field remains rare. One reason is that we lack designs and measurement approaches that appropriately meet the challenges of causal inference in the context of teacher education programs. This article provides a framework for how to fill this gap. We first outline the difficulties of doing causal research in teacher education. We then describe a set of replicable practices for developing measures of key teaching outcomes, and propose causal research designs suited to the needs of the field. Finally, we identify community-wide initiatives that are necessary to advance effectiveness research in teacher education at scale.

More →

Jing Liu, Susanna Loeb, Ying Shi.

Classroom teachers in the US are absent on average approximately six percent of a school year. Despite the prevalence of teacher absences, surprisingly little research has assessed the key source of replacement instruction: substitute teachers. Using detailed administrative and survey data from a large urban school district, we document the prevalence, predictors, and variation of substitute coverage across schools. Less advantaged schools systematically exhibit lower rates of substitute coverage compared with peer institutions. Observed school, teacher, and absence characteristics account for only part of this school variation. In contrast, substitute teachers’ preferences for specific schools, mainly driven by student behavior and support from teachers and school administrators, explain a sizable share of the unequal distribution of coverage rates above and beyond standard measures in administrative data.

More →

Cassandra M. D. Hart, Brian Jacob, Susanna Loeb.

A common rationale for offering online courses in K-12 schools is that they allow students to take courses not offered at their schools; however, there has been little research on how online courses are used to expand curricular options when operating at scale. We assess the extent to which students and schools use online courses for this purpose by analyzing statewide, student-course level data from high school students in Florida, which has the largest virtual sector in the nation. We introduce a “novel course” framework to address this question. We define a virtual course as “novel” if it is only available to a student virtually, not face-to-face through their own home high school. We find that 7% of high school students in 2013-14 enroll in novel online courses. Novel courses were more commonly used by higher-achieving students, in rural schools, and in schools with relatively few Advanced Placement/International Baccalaureate offerings.

More →

Thomas S. Dee, Susanna Loeb, Ying Shi.

Major philanthropic initiatives that incorporate features of venture-capital practices have become increasingly prominent, particularly in K-12 public education. In this study, we provide empirical evidence on the reach, character, and impact of the Broad Superintendents Academy, a prominent and controversial venture-philanthropic initiative designed to transform leadership in the nation’s largest school districts. Using a novel dataset on all Broad trainees and a linked panel data set of all large school districts over 20 years, we find that Broad superintendents have had extensive reach (e.g., serving nearly 3 million students at their peak). We also show that, within districts that hired Broad trainees, Broad superintendents were 40 percent more likely to be Black than their non-Broad peers, but also had tenures that were 18 percent shorter. Panel-based estimates provide evidence that Broad-trained leaders had no clear effects on several district outcomes such as enrollment, school closures, per-pupil instructional and support-service spending, and student completion rates. However, Broad-trained leaders initiate a trend towards an increased number of charter schools and higher charterschool enrollment.

More →

Min Sun, Alec Kennedy, Susanna Loeb.

School Improvement Grants (SIG) represent one type of governments’ capacity-building investment to spur sustainable changes in America’s persistently under-performing public schools. This study examines both short- and long-run effects of the first two cohorts of SIG schools from two states and two urban districts across the country. Using dynamic event analyses, we observe that SIG showed larger effects in the second and third years of the intervention than the first year on 3-8th grade student test scores—a pattern of gradually increase over the course the intervention. These positive effects are largely sustained three or four years after the funding ended. In high schools, the SIG effects on 4-year graduation rates were steadily increasing throughout the period of six or seven years after the initial start of the intervention. These patterns of SIG effects mostly apply to each of the four locations, but the magnitude of effects varies across locations, suggesting differential implementations. Moreover, SIG effects on students of color or low-socioeconomic students are similar to, and sometimes a bit larger than, the overall SIG effects. We also conduct a variety of sensitivity and robustness checks.  Lastly, we discuss the policy implications of our findings on states’ continuing efforts of transforming public organizations and building their long-term capacity for better performance.

More →