Chapters
Shownotes Transcript
Whoa. Landing an account this big will totally change my landscaping business. It's going to mean hiring more guys and more equipment and new trucks for the new guys to drive the new equipment in. I don't know if I'm ready. You can do this. And Ford Pro Fence Simple can help. Our experts are ready to make growing pains less painful for your business with flexible financing solutions that meet the needs of your business today when you need them. Get started at FordPro.com slash financing.
Ryan Reynolds here for, I guess, my 100th Mint commercial. No, no, no, no, no, no, no, no, no. I mean, honestly, when I started this, I thought I'd only have to do like four of these. I mean, it's unlimited premium wireless for $15 a month. How are there still people paying two or three times that much? I'm sorry.
I'm sorry, I shouldn't be victim blaming here. Give it a try at midmobile.com slash save whenever you're ready. $45 upfront payment equivalent to $15 per month. New customers on first three month plan only. Taxes and fees extra. Speeds lower above 40 gigabytes. See details. ♪
During the Obama administration, school choice was a central part of the national policy discussion. Political and policy debates between teachers' unions, parents, students, policy wonks dominated, with liberals usually opposing school choice reforms and conservatives supporting them. That conversation is less central now, at least in Washington. But slowly and without much fanfare, the school choice movement has been racking up political wins across the country.
And it's not just limited, targeted voucher programs. Activists have gotten some states to even allow parents to remove their public school dollars from the system and use them for private school or even homeschooling. But I have a question. Is it possible to get some of the benefits of school choice within the public school system?
This is Good on Paper, a policy show that questions what we really know about popular narratives. I'm your host, Jerusalem Demsys, and today we're exploring just that question through an interesting paper that I don't think has gotten enough attention. One of the great parts of my job is when I randomly come across an expert on a topic I'm less familiar with, and it opens up a whole new line of inquiry and discussion.
A couple of years ago, I was in Chicago and I met today's guest, Chris Campos. He's a labor economist at UChicago who focuses on the economics of education. And the research project he told me about then has been in the back of my mind ever since. Campos is an L.A. native, and in 2012, the Los Angeles Unified School District, or LAUSD, tried something new.
Given the demand from parents for more school choice, LAUSD decided to run an experiment. What if they gave some high school parents their pick of a few different public high schools? School choice, but within the public school system. What would that do for their students? How would that change the schools? Chris was able to study this change closely, and his findings have really interesting implications for the future of public schools in America.
Chris, welcome to the show. Thank you for having me. Excited to be here. Yeah. So we're going to talk about education and we're going to get into a really interesting paper that you wrote about Los Angeles. But
I feel like every time I start any kind of conversation about education or schools or making schools better, I feel like we just start at like what even it means to say that something is a good school. And like, how do economists think of it like you and how do parents think of it? So just bare bones, like how do we define what a good school is, what a good school does?
That's a fantastic question. And so I think economists, which is kind of what I am by training, we tend to look at how students learn over time and in particular how much a particular school may contribute to student learning over time. So sometimes this is referred to as academic growth, achievement growth. And so statistical measures that capture how much schools contribute to student learning is what economists tend to use as their measures of school quality.
Now, with that said, test scores aren't the only things that schools affect. You know, schools affect a host of non-cognitive outcomes as well. And so we could think about alternative measures, which are also kind of capturing how schools affect these alternative measures. For what I'll be talking about today, it'll primarily be focused on learning-related or achievement-related outcomes. Thank you.
How do parents think about it? Because I realize that sometimes I think about like when people tell me, oh, my kid like goes to a good school, doesn't go to a good school. Like, do I think that they have like learned whether or not the educational systems and priorities of their elementary school are causally increasing their students ability to attend college or are they saying something else?
Yeah, that's something that I've kind of done some work on in separate papers. If you just survey parents, and a lot of researchers have done that, and you ask them what they prioritize when they're choosing schools, it's a pretty robust finding that most parents will tell you that they're concerned about safety. Safety is at the top of the list, whether it be low or higher income parents, that's what they'll say. Achievement tends to come next. But if you start trying to distinguish the difference between an achievement level or a growth metric, parents aren't always able to tell the difference between the two.
And when parents are sort of thinking about like when they're after safety and they're trying to assess achievement or whether or not their kid's school will be good, like what heuristics are they using? Are they like looking at online databases or are there like other heuristics they're looking for, searching for? Right. So I think there's these popular platforms such as greatschools.org that parents tend to heavily rely on.
And in those platforms, they'll rank schools based on a host of factors. And some of these include growth metrics in some states where that data is publicly available. And other locations, it'll include just information about average achievement at schools.
But anecdotally, you know, some parents that don't have access to these platforms or are unaware of them tend to just rely on heuristics such as, you know, where do high income kids go to schools? And they'll tend to associate kind of like a school that has high income students as like a higher quality school, or they'll use kind of proxies such as race as well to kind of identify good schools. That doesn't necessarily mean those are schools that are
most causally improving student learning, but these are heuristics that some parents tend to rely on when choosing schools if they find it challenging to observe these notions of school quality.
So part of this conversation is sort of based around parent choice. Like even when we're thinking about the classic view of how people pick their schools, where they move to a neighborhood and they kind of look for the schools when they're in that moving process. There's obviously an element of choice in there. But increasingly choice in the American public school system has been a range of things from charter schools, education.
And then also being able to use state dollars for homeschooling in private schools is increasingly normal. And that has been part of this broader movement that giving parents this option to pick where their kids go to school is better because it allows them to kind of make the determination for their own family and discriminate between different educational opportunities. I know this is like a very big literature here on school choice, and I'm asking you to oversimplify for us. But what
What do we know about how expanding school choice in these ways affects student outcomes? Like what's sort of the lay of the land right now about that shift in the American educational landscape? So the original school choice reforms advocated by Milton Friedman in the 1950s and 1960s are focused on educational voucher programs.
In more recent decades, the 1990s to start in particular, there's been this rapid rise in charter schools. Voucher schools are private schools while charter schools are still publicly funded but privately managed. Both types of choice schools tend to compete with incumbent school districts for students.
But even within public school districts, there's still a substantial amount of school choice. In school districts with unified enrollment schemes, families have a substantial amount of choice. In others, they may have access to magnet programs or other programs that allow them to enroll in schools that are not their own schools. So when it comes to evaluating these reforms and what we know about their effectiveness, there's different perspectives one can take.
To begin, one can ask, do the students and families who exercise choice, whether it be the vouchers or the charter schools or the magnet programs, do they benefit from that? And so there's a host of studies studying these so-called participant effects. And I'll start with vouchers. And I'm going to focus on lottery-based evidence.
And kind of my overall sense of the literature or read of the literature is that there isn't strong evidence that vouchers causally improve student learning. So we can think about the Washington DC Opportunity Scholarship Program that produced no test score impacts. The study was roughly 2,300 students where 60% of them were randomly offered a voucher program. You can follow these kids over time and just compare those that were offered vouchers to those that were not and not finding much of an impact on test scores.
three voucher programs in New York City, Dayton, and Washington, D.C. This is also kind of quasi-experimental design. They find no average impacts on achievement. And there's this other popular voucher program, the Louisiana Scholarship Program, which found substantially large negative impacts on academic achievement as captured through test scores on English and language arts and mathematics.
So, you know, most lottery-based studies of voucher programs in the United States tend to find no measurable impact. This Louisiana study is one that finds really large negative impacts. So that's participant effects for voucher programs. We can think about the same types of studies when it comes to the charter literature. When it comes to charter schools, particularly in urban areas, there's numerous studies that have shown significant positive impacts on academic achievement, particularly in math and English and language arts.
And they've been particularly beneficial for lower performing students, non-white students, low income students, and students with disabilities. But most of the findings in this literature come from large urban areas and they cover very few states. So a lot of the most compelling evidence comes from Boston. Boston is not necessarily the most representative city in the United States. So there's limited generalizability in the results we have in the charter literature.
When it comes to market-level effects or studies of how choice programs affect the overall market, those are a bit more elusive because they're harder to study. And so this kind of segues right into kind of what I've been able to do in my zones of choice study, where they introduce centralized assignment systems in roughly 40% of the district, 40% of high school students, and the other 60% remain in the status quo with neighborhood-based assignment programs.
And because there's this partial coverage of the Zones of Choice program within LAUSD, this allows me to get at these market level effects. And sorry, can we back up a quick second here? Because I think that like the L.A. setting is really interesting. And I think that giving a lay of land is going to be helpful here. So Los Angeles, I mean, it is a highly Hispanic school district. I mean, at this point, it's like 70, nearly 75 percent Latino and Latino.
When they decided to do the zones of choice system, the ZOC system, which is what you're talking about, what exactly is that? What was the context in which they were making that shift? Right. So the ZOC system,
or the Zones of Choice expansion that happened in 2012 was predated by a pilot, which was called the Belmont Zone of Choice. And the Belmont Zone of Choice, like organically formed after this large school construction program that led to many new schools in the school district due to overcrowdedness. A lot of these new schools did not have attendance zone boundaries like most other schools in the district. So they were kind of just floating around.
And at the same time, there's this increased demand for charter schools. So a lot of families were leaving to charters. And the charter enrollment share as of like 2023, if I remember correctly, is like roughly 36%. So the charter school sector has a big presence in LAUSD. But going back to 2007, it was still kind of on the rise.
And so school district administrators sensed that families kind of had this inherent demand for more options. And so they experimented with this pilot, the Belmont Zone of Choice, where they kind of clustered in some of these new schools, some existing traditional public high schools that were part of the district, into a larger zone of choice. And within this zone of choice in the Belmont area, families were allowed to kind of enroll in any one of these schools.
And so families could apply to different schools. And if there was oversubscription, a lottery would determine who got assigned to what school. But in this era, this is going back to 2007, it was very informal. There aren't really many records of how assignment was done during this era. And if we fast forward five years, many of the same issues the district was facing with kind of charter competition, not sure what to do with these extra schools that were just around the school district,
They kind of decided to expand the program and they created 16 zones of choice. And these are like geographically bounded areas? That's correct. Okay. So instead of a family having one neighborhood school, they're now going to have several nearby options, but their options are still going to be restricted based on the address. And how did they decide which schools to include in the zones of choice? I know you said that basically like how...
Half of L.A. was split up into these zones and the other half kind of remained in the existing neighborhood school system. Right. So I think if we look at where these zones are concentrated at, most of them are going to be in South, Central and East Los Angeles, some in the San Fernando Valley. But most of them are going to be concentrated in these regions of the district that are relatively disadvantaged.
Most of the census tracts that are part of the Zones of Choice program are in the bottom quartile of the median income distribution in Los Angeles, as measured in the 2010 census. So really disadvantaged neighborhoods.
And so from one perspective, they were trying to introduce choice in sectors of the district where there was a lot of charter competition. Another perspective is if you try to change the attendance zone boundaries of families in West LA, that's nearly impossible because you're going to- And West LA is like the richer part of LA. The richer part of LA, exactly. You're going to have a lot of parents that are going to go to the school board meetings and complain about this and make the policy nearly impossible. Right.
So you're going to see that most of these zones are going to be in these disadvantaged neighborhoods where maybe there's not as much representation during the school board meetings. And so the individual that was tasked with creating these zones of choice, I think he was a bit strategic in choosing neighborhoods where families would be a bit more amenable to the creation and molding of these new zones of choice. Okay. So just to describe kind of the process by which this works, right? So instead of your neighborhood school, you have now up to, and this is just high schools, right? That's right.
Yeah. So you have now like three or five like different high schools that you can pick that are all kind of within a geographically bounded area. And then I guess like when people are finishing middle school, they get to apply or is it just they just rank their preferences or it's a lottery system? Like how does it work?
So every fall at around September, parents are informed about the Zones of Choice program in the case they had not heard about it. These are parents with children that are in eighth grade.
And then the school district has a zones of choice office and administrators at that office start arranging and scheduling numerous information sessions where they communicate features of the program to parents. Schools organize like open houses where they kind of like pitch themselves to students and kind of like try to make the best case for why they should go to that school or enroll in a given program.
And then there's a deadline, usually November 15th or mid-November, where parents are required to submit an application, ranking every option in their zone of choice. And they have an incentive to submit an application because if they do not submit an application and they're part of the Zones of Choice program, they may be assigned a school that is not necessarily one that
And what percentage of people submit applications? Is it high? From what I've looked at in the data, it's like nearly everyone. Okay. Yeah, so for the most part, everyone submits an application. Very few people don't. Conditional on you being part of the district in eighth grade. I'm sure there's many families that are not part of the district in eighth grade, and so we never really observe them in the data, but...
And so then they submit an application and then there's some centralized algorithm that then allocates students to schools based on their preferences and their priorities. And priorities tend to be geographic, sibling-based, but outside of that, there's no additional priorities.
There's no screening mechanisms in this particular setting. This is in stark contrast to a place like New York City, where screening is particularly common, but not in the zones of choice setting. Okay, great. And so what did you find? Like, how did creating these kind of like small educational markets affect school quality?
Right. So because roughly only 40% of the district gets these zones of choice programs or zones, and the other 60% does not, you can just compare trends between affected neighborhoods or students to unaffected students or neighborhoods. And then that differential change before and after the policy is expanded in 2012 informs us of how the overall policy affected student outcomes.
And so before I get to the results, let me just tell you a bit about how ZOZ students were doing kind of before the program expanded. So as ZOZ students are entering high school before the program comes into place, they're performing roughly 21 to 23% of a standard deviation, more poorly on standardized exams. You may ask, like, what is a standard deviation? So we can start by asking ourselves how much students typically learn in a year between grades 7 and 8.
Taking a number from an existing paper, that amount is roughly 26% of a standard deviation. And this allows us to translate that 23% of a standard deviation incoming achievement gap that ZOZ students face to roughly 159 learning days, given that a school year has 180 days. So one way to interpret that 23% of a standard deviation incoming achievement gap is that ZOZ students are
are performing on standardized exams, roughly equivalently to just having learned 159 less days than non-ZOZ students. That's an alternative way to view the incoming achievement gap. So, ZOZ students are relatively disadvantaged, living in relatively disadvantaged neighborhoods in LA County, and also performing more poorly on academic standardized exams. We fast forward seven years,
And the program leads to roughly 16% of a standard deviation improvement on standardized exams. If we translate that to learning days, it's roughly 139 additional learning days. Wow. So it nearly eliminates that incoming achievement gap. We can also look at college enrollment, and we find that college enrollment increases by roughly 5 percentage points. And that improvement in college enrollment is sufficiently large that eliminates the pre-ZOZ college enrollment.
gap. Okay, so basically the students who are in the ZOC schools, which you explained, like, you know, these are the parts of LA that are disadvantaged. They were less likely to have kind of backlash from rich parents. These students are now basically closing the gap between themselves and the higher income areas of Los Angeles when it comes to learning days and college enrollment. That's right.
Wow. It's exactly that. It doesn't fully close the gap for achievement, but it gets very close. And for college enrollment, it kind of flips and goes the other way. So it's essentially closed. That doesn't mean that inequality has fully gone away in Los Angeles. Schools are still vastly unequal. And there's a lot of variation in terms of student outcomes.
But at least the ZOZ, non-ZOZ gaps have essentially closed as of 2019. And specifically, we're talking about test scores in specific areas or what, when we say closing achievement gap, what are you looking at? Test scores, standardized exams, math and reading or English and language arts and college enrollment. And so if we then look at college enrollment, you may ask like what colleges are ZOZ students getting nudged into? When we look at that, we find that a majority of these students are going to California State University campuses and there's
Really not much of an impact on community college enrollment and if anything, kind of a diversion away from private schools. Were you surprised by this, like the size of this effect size? Yes, definitely I was.
I was not expecting it. This was initially supposed to be a different paper on how residential decisions responded to the program. But I had the data, so I kind of went ahead and looked at kind of what happened to achievement. And I found these results, and then I spent roughly two months trying to kill the results because I just, it shocked my prior. But yeah, that's kind of how research goes. All right. Time for a quick break. More with Chris when we get back.
The best journalism does more than just give you information. It sets you up to learn, to think critically, seek out the truth with an open mind. This is what The Atlantic does. It teaches you to question yourself, seek out new perspectives, separate fact from fiction.
Now, in this back-to-school season, for a limited time, you can get $20 off when you give someone an Atlantic gift subscription. Start them off on a lifetime of good learning at theatlantic.com slash learning.
Back to kind of the effects that you were able to find in your research, I wanted to ask, did the composition of schools, is that what's driving this? Like, is the composition changed from when the ZOC started such that now there are more higher performing students just coming into the zones of choice areas?
So I was very concerned about this just because kind of having a decent understanding of L.A. County, the zones of choice neighborhoods have been gentrifying substantially in the past 10 years. And so I was concerned that a portion of these effects were going to be driven by kind of this gentrification, the fact that these zones are now receiving relatively higher income people than before the policy expansion happened.
When I look at that in the data, I don't find evidence of any differences and changes in student composition between ZOZ and non-ZOZ schools. And I think this is due to the fact that although these neighborhoods are gentrifying, the people that are gentrifying these neighborhoods either don't have kids yet or if they do, they're very young. And so they're not going to these high schools.
So although there may be some kind of change in the composition of these neighborhoods in terms of amenities and things like that, that comes with gentrification, I'm not finding it directly impacts the results in the paper. Okay. And then also within the zones of choice, is there compositional sorting such that like now you're getting higher performing students kind of just sorting themselves into better schools and that's what's getting the benefits? Okay. When it comes to how students were sorting within the ZOZ,
One other perhaps surprising finding that I find in this particular setting is that I can look at the types of choices parents are making when applying to schools because they submit these rank-ordered preference lists to the district that then determines the assignment of students to schools.
And then with these rank-ordered preference lists, I can get measures of school popularity. And then I can just ask, what best predicts these measures of school popularity? Is it school quality? Is it student composition? Or other school factors? Like, what is most predictive of demand in these particular markets?
And I'm finding that school quality or these notions of value added that economists tend to use are most predictive of demand in the ZOZ market. And so what this is saying is that families are providing schools in the ZOZ markets incentives to care about factors that contribute to student learning because that's what's driving their demand.
And it's not that parents are selecting based on the composition of kids at a school because that would produce alternative incentives where schools wouldn't necessarily care about contributing or investing in factors that contribute to student learning. They would instead be investing in things that would attract high achieving students.
And so because I'm finding in the data that parents' choices strongly lean towards higher value-added or higher quality schools, this is providing schools the incentives to kind of invest in these things that contribute to student learning, which is then contributing to the overall impacts I'm finding in the study. There is a 2020 AER paper about whether parents value school effectiveness. And they're looking at like a quarter of a million applicants to the New York City high school system.
And they basically find that there's like no evidence that parents are – that households are kind of responding to causal school effectiveness. And they find that people are mostly looking for peer quality. And, you know, like as we talked about earlier, like that's usually kind of just measures of whether it seems like there are higher income students or there are –
uh, whiter students in the schools. And that's how they're kind of deciding whether or not to, um, you know, send their kids there or where they think that school is a good school. And so what's interesting about the design of your study, what's the design of this Los Angeles school system, uh, uh, zones of choice is that because it's so segregated, the zones of choice, you can't actually, um, discriminate based on race or on the, um,
income of your peers by that much. And so you have a system where parents are not able to default to that heuristic and then they're able to then maybe make decisions based on or then they choose to make decisions based on causal effectiveness. So just, I guess, talk
a little bit about that for me because I think that's kind of like a weird finding. It's like people want to discriminate based on these things and like you have to take it away from them in order for them to like care more about causal stuff or is there some other way I should be thinking about this? Right. Yeah, the fact that families are selecting schools based on schools' causal effectiveness is somewhat surprising of a finding here.
And so, as you point out, these zones are extremely segregated. At the time of the study, 88% of the students in the ZOZ are classified as Hispanic by the school district. This is well above 67% of Hispanic students elsewhere in the district at the time.
And so, yeah, exactly as you point out, within a zone, you know, when a parent is choosing a school, they can't find the school that has all the rich kids or they can't find a school that has all the white kids. And so it doesn't mean that in a place like New York City, families are necessarily discriminating, but they're using these heuristics that make it somewhat easier, at least to them, to identify these effective schools. When the ZOZ setting, because they don't have this easy heuristic, they're forced to go find other things to differentiate the schools.
And when they do that, it turns out that whatever it is they're then kind of choosing to select schools on tends to be something that's going to be more correlated or more strongly associated with schools' causal effectiveness. At least that's kind of what I think is going on in the zones of choice. Because at the core, I don't think families are necessarily observing school value added and kind of making decisions based on that in this setting. But they're using something else. They're doing something else.
because they don't have these proxies for quality. And so, I mean, I guess that's part of my question here, right? Because what would they be looking at, right? Because I kind of lean in your direction that people don't want to just say, like, I want to send my kids to the place where white kids are. They're trying to say, like, I want to send my kids to the place that, you know, maybe has high investment or whatever. And they're using it as a proxy, as you say. But if without that, I mean, is it the case that there are actually good proxies that more
places could be using to actually evaluate whether or not a school is making their would make their kids better off. Right. So in the data that I have, it's really hard to kind of
figure out what that thing is. I think anecdotally, at least back when I was a graduate student and I was going to some of these information sessions, I think parents do get a lot from talking to the principals in these open houses where several principals were present or in these information sessions where they get a lot more information that you could associate with soft skills or things like that.
Things like that that aren't necessarily objective measures of quality, but that are nonetheless going to be correlated with these objective measures of quality. And I think in a place like New York City, where a family has hundreds of schooling options, it's impossible for them to go and learn about all of these hundred options. In the ZOZ setting, families are required to learn about anywhere between two to five campuses. So it's a bit more manageable.
And so I think because you eliminate these proxies for quality such as race or income, but at the same time you make the choice that's sufficiently small, you make learning about your options sufficiently feasible. And so everyone kind of does pick up on other things. It's unclear what those things are, but I think it's a combination of kind of eliminating the sorting on race and income and making these choice sets sufficiently small that it makes it manageable for families to learn about their options.
And I know you said you looked at survey data about students, but did you do any, like when you were at these fairs or information fairs, were the kids present too? Like were you able to talk to them or any qualitative stuff on your interactions with them? Nothing systematic, to be honest. It's on my to-do list to do a more qualitative study with the principals to kind of just directly ask them how they kind of perceive the program, how they engage and interact with it. But that's been held up.
So when we met a couple years ago and you were telling me about this study that you were working on, I think...
At first, I didn't fully grasp that this was only happening within a segregated school system and that the effects were not just happening because you were desegregating. I think that was my first essential assumption. Like, oh, yeah, this will happen because you have moving to opportunity stuff that's going on here where people are now able to go to the better neighborhood schools and you're then desegregating and sending the kids from the higher income neighborhoods also to the lower income schools. And that's not what's happening at all, which feels like...
I don't know. It feels kind of odd to me because I think my expectation is that like the large gains that we would expect to see from schools is going to happen if we can get desegregation. I mean, does this change your sense of whether desegregation is required to get kind of better outcomes for students in these high risk populations? Yeah.
I don't know if it's moved my prior on that necessarily. I think what this study does do is serve as a case study into like where the school choice reforms or this implementation of these centralized assignment systems can be effective and some of the features of it that potentially facilitate the effectiveness of the program. I do still think that there is a value to desegregating schools. There's a ton of research supporting that.
those ideas. But it is a bit strange, as you point out here, that, you know, there's this potential trade-off where, you know, you could have a very successful school choice program, but perhaps you need to eliminate kind of the sorting on race and income and create this homogenous kind of market to then facilitate parents selecting effective schools, to then incentivize schools to invest in things that contribute to student learning,
But it's unclear from a policymaker perspective if you want to have these kids and their K-12 education in highly segregated schools.
But at the same time, they're experiencing substantial improvements in achievement. They're going to college more often. So it's unclear how we should weigh those two factors. To me, I'm very interested in the mechanism to get these better outcomes. Like, what are the schools actually doing then? If they are seeing these increased college enrollment and better test scores, like, what's actually happened to get that to happen? You know, I'll give you something that comes out of the data, but it's also not going to be the most convincing part of the paper.
But, you know, this is always a weakness of most empirical papers in the social sciences. So what did these schools do differently that could lead to such stark changes? Given the data that I have, I have two primary sets of findings, and this is all suggestive. The first relates to changes in schooling practices. So I find a sharp increase in suspensions, roughly 31% increase in suspension rates in ZOZ schools relative to the pre-ZOZ mean, and a sizable drop in absenteeism.
And so these two findings are consistent with a setting with stricter disciplinary practices and increased expectations. And so these two practices are part of a broader bundle that tends to be referred as like the no excuses approach to urban education. At the same time, you know, I also find perhaps reassuringly that ZOZ students are also experiencing increases in their SAT scores. And so
some of these learning gains that we're capturing in these standardized exams do seem to also be materializing in the SAT that the students that are going to college are taking. And so this is going to mediate this increase in college enrollment rates among ZOZ students.
Right. So these are the two things that I'm finding, at least that stand out the most as substantial changes within the school. I'm interested in this because, I mean, well, I guess firstly, I think that there is a lot of people have a kind of expectation that when they see that a charter school, for instance, has led to improved outcomes, that what's happening is either they're screening on the front end or they're just expelling kids on the back end. Did you find evidence that they're kind of selectively expelling low performing students? Absolutely not in this particular setting.
And another kind of concern that people have in the charter literature is that these potential high suspension rates potentially impact student well-being. And so in LAUSD, since 2010, they run this school experience survey where they survey essentially every child in the school district. And they ask them about their happiness and their overall well-being. And I'm finding no detrimental impacts on that.
on student happiness or well-being, and if anything, improvements in those measures. And is that relative to the non-ZOC students? That's right, relative to the non-ZOC students.
students. Okay. And so, you know, the no excuses model and what you just mentioned with kind of in a harsher disciplinary system or at least a stricter disciplinary system in schools, this has been studied extensively in the literature. What is, does this push against or is this kind of affirming what we already know about how this affects outcomes?
Right. So I think like the charter literature made like, like there was a lot of progress on that in the 2010s and has kind of stagnated since. And I think like the research kind of ended and said there's successful charter schools and urban locations and the no excuses approach tends to be very predictive of positive charter impacts.
In recent years, the no excuses approach is rebranded to high expectations, high support because of some of the reasons you mentioned earlier. It's unclear if what I'm finding in this paper necessarily links back to that because, you know, these are public schools. These aren't charter schools. I'm just finding evidence of a similar change in a schooling practice that's been associated with positive learning impacts in other settings. So I think that's as far as I can take it here. Yeah. But yeah.
So a lot of this, though, like this change kind of that happens, it's not premeditated.
predetermined that it would. I could imagine a world in which these zones of choice pop up, but like, it's not like Los Angeles City Schools was saying, we're going to take away your funding if you're like on the bottom end of people's prioritization. So like what, and I would imagine even before zones of choice, like there were a lot of people who were working in schools who cared a lot about making the kids better off. So what made them actually, what was the
push incentive to actually make them change any processes inside the school, why would Zones of Choice actually do that? Right. That's a great question. And sometimes people ask the same question in a different way. And they ask me, like, are principals or teachers lazy if you don't have these school choice programs? I think the answer to that is no. I think I've never talked to a principal whom I doubted cared about the welfare of their students. They're all fantastic and deal with a host of different problems. And in many ways, they're superheroes.
And so I can talk about a scenario where a well-meaning principal would naturally respond to this policy, and it wouldn't really imply that they didn't care before the policy. So in the status quo with neighborhood-based assignment, principals have a somewhat fixed set of students they educate every year because neighborhoods determine enrollment flows.
In the zones of choice area, it's not obvious what kids the principal is going to receive because some kids may opt to go to a different school that is not their neighborhood school now that we've expanded kind of the attendance zone boundary. If we introduce the fact that although LAUSD does not use a student-centered funding formula, as you pointed out, school funding levels are still positively associated with enrollment.
Therefore, any loss of enrollment could lead to the loss of a teacher, a counselor, a nurse, and et cetera. And in this scenario, any well-meaning principal is going to naturally care about the enrollment because it could potentially affect the number of teachers the school has, the counselors, nurses. And so they're going to naturally care about having higher enrollment. And whatever is going to contribute to more enrollment levels matters.
is what the principals are going to be responding to. And so both in the pre-ZOZ era and the post-ZOZ era, principals are equally caring about their children, but just now they have to respond to different incentives to ensure that they kind of sustain the same level of education for their students.
I guess to me, though, because like that incentive to prevent low enrollment is existing whether or not you have the system or not. Right. Like in a school system that doesn't do zones of choice and just has people neighborhood schools, like people will move away from really, really bad schools. Like it's obviously harder and it's over like longer timelines. People don't just move every year.
But, you know, you do see declining enrollment and declining funding and, you know, losing teachers and guidance counselors in schools that are doing very, very poorly. So is it that they're just worried about losing enrollment or is it that like now they're actually kind of being ranked in like a very obvious way? Yeah.
Definitely. So career concerns are probably definitely a thing that principals are thinking of. I can hypothesize here most, but if you just follow the trajectory of many school district administrators, you'll see that many start off as teachers, they become assistant principals, and they become principals, and then at some point get promoted and end up in some administrative position in the school district.
And so, you know, if they do have career concerns, obviously being kind of the highest performing principal in the zone of choice is probably something you're going to embellish on your resume or make sure it's very salient on your resume. And so once you kind of introduce these additional career concerns, it's an additional incentive principals may have to kind of really perform really well once you create these zones of choice markets in comparison to the status quo where it's just neighborhood based assignment.
It's funny, though, because it kind of like there's a pretty big effect sizes you're finding. And obviously, I know that you're saying it's hard to figure out exactly what the exact mechanisms are happening here are. But if it is that principals are now changing school policy as a result of this concern about declining enrollment, I mean, yeah.
That means principals, I mean, have a huge impact, like their individual person has. And I guess maybe it's also like other administrators, but a handful of people at a school can vastly change the outcomes without changing funding, without changing who the teachers are, without changing who the students are. I mean, that's remarkable. Is that surprising to you? Are you worried about the durability of that?
No, yeah, I think that's a great point. And I totally agree. I mean, if you ever kind of just go to like one of these public schools or just any public school in general, like you really see and get a sense that the principal really sets the tone and the culture of the school. So they do have a lot of power to kind of change just the overall environment. So that I definitely hold heartily believe. And yep, I also agree that, you know, this kind of
One interpretation of the study is that principals do have enormous sway in changing the educational trajectories of their children without significantly altering funding structures and things like that. Gotcha.
Well, this is a great place for our final question. What is something that you thought was good on paper but didn't pan out how you expected? Right. This is a fascinating question because I think no matter how good we think something is on paper, there tends to always be a dissenting opinion somewhere. So in that spirit, let's go with the topic of school closures or extended remote instruction during the pandemic. Okay.
and in particular from fall 2020 onward. So, you know, at the time, there was mounting evidence, even from governments such as Switzerland, suggesting that transmission was not as high among children. But school administrators in the United States were balancing the potential health concerns of teachers, students, and other staff against the educational implications of extended lower-quality remote schooling. It was a tough position to be in.
With the growing yet mixed evidence suggesting children were not as contagious, some advocated for opening schools. If we take the evidence at the time, keeping schools open seemed to be good on paper. But this position stood in stark contrast with the lived experiences of many disadvantaged families, perhaps in multi-generational households.
On the off chance one of their children gets COVID and it spreads within the family, that could lead to devastating consequences. And so from their vantage point, the health risk of parents, students, staff, and teachers were vastly greater than the short-run learning loss to students. And so to district officials with sizable shares of disadvantaged families, it was evident that keeping schools closed or in remote status was good on paper.
And so at the time, both perspectives were still clouded by a lot of uncertainty. If there would have been a new COVID variant that children did effectively transmit, the picture would have changed. School closures may have proved to be a good thing.
But that turned out to not be the case, and the additional learning loss induced by extended school closures was immense. So existing research shows that school districts that remained in person experienced smaller declines in learning, as captured by passing rates on standardized exams. And while the pandemic seemed to have widened achievement gaps, other existing work shows that those gaps were not as wide in districts that remained in person. And so that's my example.
And did you think it was good on paper? Like at the beginning of the pandemic, were you like, yeah, this is reasonable. We should probably close down schools. Yeah, I would go back and forth. But I think probably leaning more towards schools
keeping them closed or keeping them remote a bit longer. It's kind of like the mask debate early in the pandemic, right? There's so much uncertainty. Why don't we just, why aren't we just all risk averse and wear masks and figure out later if it was effective or not? That's kind of how I felt about school closures or openings in fall of 2020. I was like, there's still too much uncertainty. But the evidence was mounting that kids weren't transmitting COVID as much as adults. Yeah.
Well, Chris, thank you so much for coming on the show. I really enjoyed this. This is great. Thank you so much. This was a lot of fun. Yeah, for sure. Now you'll have to tell me about a study you're doing now and in two years. I'm like, I have the review deadline of a top tier economics journal. Like I'll come back to you in two years.
Good on Paper is produced by Janae West. It was edited by Dave Shaw, fact-checked by Anna Alvarado, and engineered by Erica Huang. Our theme music is by Rob Smirciak, Claudina Bade is the executive producer of Atlantic Audio, and Andrea Valdez is our managing editor. And hey, if you like what you're hearing, be sure to follow the show, and please leave us a rating and review on Apple Podcasts. I'm Jerusalem Dempsis, and we'll see you next week.
There's a common perception that democracy ends with a battle, soldiers in the streets, a coup d'etat, the fall of a government. But we know that democracy can be lost one little step at a time. We've reported on it and lived through it. And when we look at America today, right now, we see a place where the slide to autocracy has already begun. It's not some distant future, it's the present.
I'm Anne Appelbaum, a staff writer at The Atlantic. I'm Peter Pomerantsev, a senior fellow at the SNF Agora Institute at Johns Hopkins University. We're the hosts of a new podcast from The Atlantic, Autocracy in America. Subscribe to the show wherever you get your podcasts.