Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Educator preparation, professional development, performance and evaluation

Matthew A. Kraft, Alexander Bolves.

We study the adoption and implementation of a new mobile communication app among a sample of 132 New York City public schools. The app provides a platform for sharing general announcements and news as well as engaging in personalized two-way communication with individual parents. We provide participating schools with free access to the app and randomize schools to receive intensive support (training, guidance, monitoring, and encouragement) for maximizing the efficacy of the app. Although user supports led to higher levels of communication within the app in the treatment year, overall usage remained low and declined in the following year when treatment schools no longer received intensive supports. We find few subsequent effects on perceptions of communication quality or student outcomes. We leverage rich internal user data to explore how take-up and usage patterns varied across staff and school characteristics. These analyses help to identify early adopters and reluctant users, revealing both opportunities and obstacles to engaging parents through new communication technology.

More →


Philip Oreopoulos.

This article takes stock of where the field of behavioral science applied to education policy seems to be at, which avenues seem promising and which ones seem like dead ends. I present a curated set of studies rather than an exhaustive literature review, categorizing interventions by whether they nudge (keep options intact) or “shove” (restrict choice), and whether they apply a high or low touch (whether they use face-to-face interaction or not). Many recent attempts to test large-scale low touch nudges find precisely estimated null effects, suggesting we should not expect letters, text messages, and online exercises to serve as panaceas for addressing education policy’s key challenges.  Programs that impose more choice-limiting structure to a youth’s routine, like mandated tutoring, or programs that nudge parents, appear more promising.

More →


Matthew A. Kraft, Nicole S. Simon, Melissa Arnold Lyon.

COVID-19 shuttered schools across the United States, upending traditional approaches to education. We examine teachers’ experiences during emergency remote teaching in the spring of 2020 using responses to a working conditions survey from a sample of 7,841 teachers across 206 schools and 9 states. Teachers reported a range of challenges related to engaging students in remote learning and balancing their professional and personal responsibilities. Teachers in high-poverty and majority Black schools perceived these challenges to be the most severe, suggesting the pandemic further increased existing educational inequities. Using data from both pre-post and retrospective surveys, we find that the pandemic and pivot to emergency remote teaching resulted in a sudden, large drop in teachers’ sense of success. We also demonstrate how supportive working conditions in schools played a critical role in helping teachers to sustain their sense of success. Teachers who could depend on their district and school-based leadership for strong communication, targeted training, meaningful collaboration, fair expectations, and recognition of their efforts were least likely to experience declines in their sense of success.

More →


Jason A. Grissom, David S. Woo, Brendan Bartanen.

High rates of principal turnover nationally mean that school districts constantly are called on to recruit and select new principals. The importance of a school’s principal makes choosing candidates who will be effective paramount, yet we have little evidence linking information known to school districts at time of selection to principal’s future job performance. Using data from Tennessee, we test the degree to which observable information about novice principals from prior to entry, including qualifications, work history information, and effectiveness in prior roles, predicts practice ratings assigned to them in their initial years in the principalship. We find that educational attainment and years of experience in other jobs hold little predictive power. Performance ratings received as an assistant principal (AP) or teacher, however, do predict principal effectiveness. Moreover, APs who previously worked in schools with highly rated principals are more likely to be effective upon transitioning into the principalship.

 

More →


Brendan Bartanen, Laura K. Rogers, David S. Woo.

Assistant principals are important education personnel, both as essential members of school leadership teams and apprentice principals. However, empirical evidence on their career outcomes remains scarce. Using statewide administrative data from Tennessee and Missouri, we provide the first comprehensive analysis of AP mobility. While prior work focuses only on AP promotions into principal positions, we also account for APs who exit school leadership and transfer to a different school. We find yearly mobility rates of 25–28%, with 10% of APs leaving school leadership, 7.5% changing schools, and 7.5–10% becoming principals. We also document a strong relationship between AP mobility and principal turnover, where higher-performing APs are substantially more likely to replace their departing principal. Principal transitions also appear to increase the likelihood that APs exit school leadership and change schools, highlighting an additional cost of high rates of principal churn.

More →


David D. Liebowitz.

Teacher evaluation policies seek to improve student outcomes by increasing the effort and skill levels of current and future teachers. Current policy and most prior research treats teacher evaluation as balancing two aims: accountability and skill development. Proper teacher evaluation design has been understood as successfully weighting the accountability and professional growth dimensions of policy and practice. I develop a model of teacher effectiveness that incorporates improvement from evaluation and detail conditions which determine the effectiveness of teacher evaluation for growth and accountability at improving student outcomes. Drawing on empirical evidence from the personnel economics, economics of education and measurement literatures, I simulate the long-term effects of a set of teacher evaluation policies. I find that those that treat evaluation for accountability and evaluation for growth as substitutes outperform policies that treat them as complements. I conclude that optimal teacher evaluation policies would impose accountability on teachers performing below a defined level and above which teachers would be subject to no accountability pressure but would receive intensive instructional supports.

More →


Andre Joshua Nickow, Philip Oreopoulos, Vincent Quan.

Tutoring—defined here as one-on-one or small-group instructional programming by teachers, paraprofessionals, volunteers, or parents—is one of the most versatile and potentially transformative educational tools in use today. Within the past decade, dozens of preK-12 tutoring experiments have been conducted, varying widely in their approach, context, and cost. Our study represents the first systematic review and meta-analysis of these and earlier studies. We develop a framework for considering different types of programs to not only examine overall effects, but also explore how these effects vary by program characteristics and intervention context. We find that tutoring programs yield consistent and substantial positive impacts on learning outcomes, with an overall pooled effect size estimate of 0.37 SD. Effects are stronger, on average, for teacher and paraprofessional tutoring programs than for nonprofessional and parent tutoring. Effects also tend to be strongest among the earlier grades. While overall effects for reading and math interventions are similar, reading tutoring tends to yield higher effect sizes in earlier grades, while math tutoring tends to yield higher effect sizes in later grades. Tutoring programs conducted during school tend to have larger impacts than those conducted after school.

More →


David D. Liebowitz, Lorna Porter.

Despite empirical evidence suggesting the important influence school leaders have on learning conditions and student outcomes in schools, relatively little is understood about the professional pathways they take into their roles. In this descriptive paper, we document the professional experiences, personal characteristics and instructional effectiveness of Oregon's principals and assistant principals between 2006 and 2019. We highlight the diversity of roles educators assume prior to entering school leadership. We find that school leaders who have prior teaching experience in tested grades and subjects do not raise student achievement at substantively or statistically meaningful higher rates than their peers. We document that female principals and assistant principals have become more representative of the teaching workforce, but that there have been almost no changes in the racial/ethnic composition of school leaders in Oregon. Finally, we observe minimal differences in female and non-White assistant principals' time-to-entry into the principalship. Our findings provide insights on potential points of intervention during the educator career trajectory to attract and develop more effective and demographically representative school leaders.

More →


Heather C. Hill, Zid Mancenido, Susanna Loeb.

Despite calls for more evaluative research in teacher education, formal assessments of the effectiveness of novel teacher education practices remain rare. One reason is that we lack designs and measurement approaches that appropriately meet the challenges of causal inference in the field. In this article, we seek to fill this gap. We first outline the difficulties of doing evaluative work in teacher education. We then describe a set of replicable practices for developing measures of key teaching outcomes, and propose evaluative research designs that can be adapted to suit the needs of the field. Finally, we identify community-wide initiatives that are necessary to advance useful evaluative research.

More →


Matthew A. Kraft, Manuel Monti-Nussbaum.

Narrative accounts of classroom instruction suggest that external interruptions, such as intercom announcements and visits from staff, are a regular occurrence in U.S. public schools. We study the frequency, nature, and duration of external interruptions in the Providence Public School District (PPSD) using original data from a district-wide survey and classroom observations. We estimate that a typical classroom in PPSD is interrupted over 2,000 times per year, and that these interruptions and the disruptions they cause result in the loss of between 10 to 20 days of instructional time. Administrators appear to systematically underestimate the frequency and negative consequences of these interruptions. We propose several organizational approaches schools might adopt to reduce external interruptions to classroom instruction.

More →