Search for EdWorkingPapers here by author, title, or keywords.
The COVID-19 pandemic led to an abrupt shift from in-person to virtual instruction in Spring 2020. We use two complementary difference-in differences frameworks, one that leverages within-instructor-by-course variation on whether students started their Spring 2020 courses in person or online and another that incorporates student fixed effects. We estimate the impact of this shift on the academic performance of Virginia’s community college students. With both approaches, we find modest negative impacts (three to six percent) on course completion. Our results suggest that faculty experience teaching a given course online does not mitigate the negative effects. In an exploratory analysis, we find minimal long-term impacts of the switch to online instruction.
This paper reports the results of a large, school-level randomized controlled trial evaluating a set of three informational interventions for young people choosing high schools in 473 middle schools, serving over 115,000 8th graders. The interventions differed in their level of customization to the student and their mode of delivery (paper or online); all treated schools received identical materials to scaffold the decision-making process. Every intervention reduced likelihood of application to and enrollment in schools with graduation rates below the city median (75 percent). An important channel is their effect on reducing nonoptimal first choice application strategies. Providing a simplified, middle-school specific list of relatively high graduation rate schools had the largest impacts, causing students to enroll in high schools with 1.5-percentage point higher graduation rates. Providing the same information online, however, did not alter students’ choices or enrollment. This appears to be due to low utilization. Online interventions with individual customization, including a recommendation tool and search engine, induced students to enroll in high schools with 1-percentage point higher graduation rates, but with more variance in impact. Together, these results show that successful informational interventions must generate engagement with the material, and this is possible through multiple channels.
The COVID-19 pandemic has been a seismic and on-going disruption to K-12 schooling. Using test scores from 5.4 million U.S. students in grades 3-8, we tracked changes in math and reading achievement across the first two years of the pandemic. Average fall 2021 math test scores in grades 3-8 were .20-27 standard deviations (SDs) lower relative to same-grade peers in fall 2019, while reading test scores decreased by .09-.18 SDs. Achievement gaps between students in low-poverty and high-poverty elementary schools grew by .10-.20 SDs, primarily during the 2020-21 school year. Observed declines are more substantial than during other recent school disruptions, such as those due to natural disasters.
Dual-enrollment courses are theorized to promote students' preparedness for college in part by bolstering their beneficial beliefs, such as academic self-efficacy, educational expectations, and sense of college belonging. These beliefs may also shape students' experiences and outcomes in dual-enrollment courses, yet few if any studies have examined this possibility. We study a large dual-enrollment program created by a university in the Southwest to examine these patterns. We find that mathematics self-efficacy and educational expectations predict performance in dual-enrollment courses, even when controlling for students' academic preparedness, while factors such as high school belonging, college belonging, and self-efficacy in other academic domains are unrelated to academic performance. However, we also find that students of color and first-generation students tend to have lower self-efficacy and educational expectations before enrolling in dual-enrollment courses, in addition to having lower levels of academic preparation. These findings suggest that students from historically marginalized populations may benefit from social-psychological as well as academic supports in order to receive maximum benefits from early postsecondary opportunities such as dual-enrollment. Our findings have implications for how states and dual-enrollment programs determine eligibility for dual-enrollment as well as how dual-enrollment programs should be designed and delivered in order to promote equity in college preparedness.
We develop a new framework for identifying at-risk students in public schools. Our framework has two fundamental advantages over status quo systems: (1) it is based on a clear definition of what it means for a student to be at risk and (2) it leverages states’ rich administrative data systems to produce more informative risk measures. Our framework is more effective than common alternatives at identifying students who are at risk of low academic performance and we use policy simulations to show that it can be used to target resources toward these students more efficiently. It also offers several other benefits relative to status quo systems. We provide an alternative approach to risk measurement that states can use to inform funding, accountability, and other policies, rather than continuing to rely on broad categories tied to the nebulous concept of “disadvantage.”
While teacher coaching is an attractive alternative to one-size-fits-all teacher training and development, the need for a large number of highly skilled coaches raises potential challenges for scalability and sustainability. Collaborating with a national teacher training organization, TNTP, our study uses administrative records to estimate the degree of heterogeneity in coach effectiveness at improving teachers’ instructional practice, and characteristics of coaches that explain these differences. We find substantial variability in effectiveness across coaches (0.2 to 0.35 standard deviations), which is close to the full effect of coaching programs as identified in other research. We also find that coach-teacher race/ethnicity-matching predicts changes in teacher practice, suggesting that the relational component of coaching is key to success.
Despite the prevalence of school tracking, evidence on whether it improves student success is mixed. This paper studies how tracking within high school impacts high-achieving students’ short- and longer-term academic outcomes. Our setting is a large and selective Chinese high school, where first-year students are separated into high-achieving and regular classrooms based on their performance on a standardized exam. Classrooms differ in terms of peer ability, teacher quality, class size, as well as level and pace of instruction. Using newly collected administrative data and a regression discontinuity design, we show that high-achieving classrooms improve math test scores by 23 percent of a standard deviation, with effects persisting throughout the three years of high school. Effects on performance in Chinese and English language subjects are more muted. Importantly, we find that high-achieving classrooms substantially raise enrollment in elite universities, as they increase scores on the national college entrance exam—the sole determinant of university admission in China.
We explore the dynamics of competitive search in the K-12 public education sector. Using data from Boston Public Schools, we document how teacher labor supply varies substantially by position types, schools, and the timing of job postings. We find that early-posted positions are more likely to be filled and end up securing new hires that are better-qualified, more-effective, and more likely to remain at a school. In contrast, the number of applicants to a position is largely unassociated with hire quality, suggesting that schools may struggle to identify and select the best candidates even when there is a large pool of qualified applicants. Our findings point to substantial unrealized potential for improving teacher hiring.
In the competitive U.S. higher education market, institutions differentiate themselves to attract both students and tuition dollars. One understudied example of this differentiation is the increasing trend of "colleges" becoming "universities" by changing their names. Leveraging variation in the timing of such conversions in an event study framework, I show that becoming a university increases enrollments at both the undergraduate and graduate levels, which leads to an increase in degree production and total revenues. I further find that these effects are largest when institutions are the first in their market to convert to a university and can lead to negative spillover effects on non-converting colleges.
School closures induced by COVID-19 placed heightened emphasis on alternative ways to measure student learning besides in-person exams. We leverage the administration of phone-based assessments (PBAs) measuring numeracy and literacy for primary school children in Kenya, along with in-person standardized tests administered to the same students prior to school shutdowns, to assess the validity of PBAs. Compared to repeated in-person assessments, PBAs did not severely misclassify students’ relative performance, but PBA scores did tend to be further from baseline in-person scores than repeated in-person assessments from each other. As such, PBAs performed well at measuring aggregate but not individual learning levels. Administrators can therefore use these tools for aggregate measurement, such as in the context of impact evaluation, but be wary of PBAs for individual-level tracking or high-stakes decisions. Results also reveal the importance of making deliberate efforts to reach a representative sample and selecting items that provide discriminating power.