Beneath A Steady Race, A Shifting America
39:23 Share

Hey, Fidelity. What's it cost to invest with the Fidelity app? Start with as little as $1 with no account fees or trade commissions on U.S. stocks and ETFs. That's music to my ears. I can only talk. Investing involves risk, including risk of loss. Zero account fees apply to retail brokerage accounts only. Sell order assessment fee not included. A limited number of ETFs are subject to a transaction-based service fee of $100. See full list at fidelity.com slash commissions. Fidelity Brokerage Services, LLC. Member NYSE SIPC.

I'm going to start a podcast called Call Him Mommy. I could get behind that. Like, I am mother. Can we use Kesha's beep beep as the background track for this podcast? Yes. Welcome to Call Him Mommy. Hello, mommy gang. Beep beep.

Hello and welcome to the FiveThirtyEight Politics Podcast. I'm Galen Druk, and we are now officially less than a month away from Election Day. And of course, many Americans are already voting. Last week, the Harris campaign got some good news in the form of the federal jobs numbers. The labor market added a quarter of a million jobs, beating expectations by 150,000 and lowering unemployment to 4.1%.

Also last week, an unsealed court filing from special counsel Jack Smith shown a spotlight on former President Trump's attempt to overturn the 2020 election. Former member of Republican House leadership Liz Cheney punctuated those details while campaigning for Vice President Harris in Wisconsin at the end of the week. And you guessed it, the VP debate did, well, nothing to move the polls. So here's what we're going to focus on today.

First, it has been a year since the Hamas terrorist attack on Israel. How has that year shaped Americans' views about the conflict, even if it doesn't seem to be shaping the 2024 presidential election? Also, how many Americans fall into the same category as Liz Cheney? She could get behind and even help legislate much of Trump's agenda. But January 6th was a bridge too far.

And the polls are currently suggesting we could see a racial realignment in this election. There may be some reasons to be skeptical, but how would it affect the electoral map if it does come to pass? And lastly, we've got a good or bad use of polling for you. An experimental polling outfit has decided to scrap real people altogether and instead poll bots that are created based on census data. Here with me to discuss it all is Director of Data Analytics, Elliot Morris. Welcome to the podcast, Elliot.

Hi, Galen. Also here with us is senior researcher Mary Radcliffe. Welcome to the podcast. Hey, Galen. Great to be here. Good to have you. So we're going to start things off with the anniversary today. So October 7th marks one year since Hamas militants launched a surprise attack on Israel, resulting in the deaths of more than 1,200 people, injuries to 6,900 others, and the kidnapping of more than 200 Israeli citizens.

The attack and Israel's subsequent retaliation in Gaza sparked a wider conflict, and in August, the AP reported that the death toll in Gaza surpassed 40,000.

Last week, the conflict escalated. On October 1st, Israeli military forces invaded Lebanon, targeting the Iran-backed militant group Hezbollah, which has been launching aerial attacks on Israel since the start of the war. In response, Iran fired more than 50 ballistic missiles into Israel. The situation risks tipping into an even wider regional war in the Middle East.

And we talked about this last week and suggested that this may not be changing Americans' calculus when it comes to the election. So today I have a different question, which is, nonetheless, how have Americans' views about the conflict in the Middle East evolved over the past year? Mary, you know, who do they blame? What do they want to see done to resolve it, even if it is not determining who they'll vote for this year?

If you look over the past year worth of data, what's interesting is there's actually a pretty clear turning point on American public opinion, which really was in January of 2024. If you look at YouGov Economist, they are a weekly pollster and they ask sort of the same questions on a regular basis. One of the questions they ask is whether Americans favor increasing, decreasing, or maintaining the same amount of military aid to Israel.

At the beginning of the war, Americans generally said that they thought we should increase military aid or maintain it the same. The number of people who said they wanted less military aid was just 20 percent of the sample. Thirty one percent said more military aid.

Right around January, though, that sort of switches. And you see the number of people thinking we should increase military aid drop by 10 to 15 percent. The number of people who say we should maintain the same amount of military aid drops pretty significantly. The most recent survey has it at just 25 percent and 38 percent now saying we should be providing less military aid. So really around January, Americans kind of got fed up with

support we were providing to Israel and thought we really should be decreasing that. You see sort of an opposite trend on providing humanitarian aid to Palestinians in the same set of surveys. Right around January, people started kind of changing their minds, thinking we should be providing more aid to the Palestinians.

rather than at the very beginning where people were a little more unsure on this question. There's a lot more unsures on humanitarian aid. Is this something specific to the conflict in the Middle East? Or is this something that is consistent across international conflicts? Because I mean, tracking the polling on Ukraine, you see that over time, Americans just decreasingly support sending military aid to Ukraine. The conflict

The contexts here are different and the politics are quite different. But is there just a certain point at which Americans become fatigued by knowing that their own country is involved in a conflict that does not necessarily touch us directly?

Yeah, that may be true. And in the case of Israel, it looks like there are sort of instigating events that really like brought that on maybe sooner than we would have seen it in other conflicts. Certainly to have decrease in support for aid so quickly from October to January, just a couple of months. And I think this is because in January is when we started to see reports of Israeli strikes in Rafah. And that got a lot of publicity and

I will say, you know, there is still support, though, for America playing a role in the conflict. If you look at a Pew Research survey from September of this year, the support for U.S. playing a role has actually increased when you frame it as a diplomatic role rather than as a financial role. So they asked what kind of role the U.S. should play in diplomatically resolving the Israel-Hamas war. You have 24 percent saying a major role, 37 percent saying a minor role. And that's

up pretty significantly from February this year. People are maybe less interested in spending our money, but certainly interested in spending our time helping to resolve this conflict or get to some sort of deal.

So I think looking at voters' priorities in this election, there's little indication that this is going to motivate how folks will choose between Harris and Trump, specifically this conflict in the Middle East. But I do wonder if some of the turmoil, global turmoil...

increases the way that Americans look at the two candidates in the sense of who could I more imagine in the Situation Room or behind the Resolute Desk or in an environment where the United States is, God forbid, attacked or for some reason gets involved in a wider war. On that question, you know, who do you trust more to handle foreign policy or national security between Harris and Trump? Who wins out, Elliot?

Yeah, so the Pew Research Center gives Donald Trump a small margin over Harris on the general question of, you know, which party or which candidate do you trust more to handle foreign policy issues? And more recently, there's a Fox News survey that asks exactly about the Israel-Hamas and, I guess, broadening crisis. And there's an even larger Trump lead, about seven points there.

I guess the question is, like, how does the Trump campaign use that and get the voters who care about this to sort of mobilize and vote for him? This has been one of the defining issues of, like, conservative online discourse over the last year. Not only Israel Hamas, but also, like, how this connects to funding for wars in general, in Ukraine being the other big example. And it's one of J.D. Vance's biggest stump speech points is that we shouldn't be involved.

The question of whether or not Trump can translate that into whatever small remaining slice of the electorate that isn't already activated by this, I guess, is the sort of relevant question. That's impossible for us to answer with the type of polling data we have, but I imagine it's a really, really small number of people.

All right, something that we will continue tracking. Last Wednesday, a federal judge unsealed a court filing from special counsel Jack Smith, which filled out more detail about Trump's efforts to overturn the 2020 election. It clarified the prosecution's position that Trump made his attempt to stay in office as a presidential candidate and not as president, meaning he would not be immune from prosecution per the Supreme Court's ruling this summer.

Much of what's in the brief we already know in broad strokes, that Trump planned to declare victory in 2020 regardless of the results on Election Day. It lays out evidence to suggest that Trump knew he had lost when he tried to assemble an alternate slate of electors who would throw out the votes of Americans in battleground states, and that Trump was aware of the dangers posed by the people he sent to the Capitol on January 6th while doing nothing to stop them. And it goes on.

I mean, given everything that we know and have frankly already discussed over the past four years on this podcast, there's really no beating around the bush. Trump rejected American self-governance in 2020. That's a fact of this election that one of the candidates tried to undo a core tenet of our constitutional republic, maybe even the most important one. So on one hand, that's a really big deal.

On the other hand, a question that we often ask on this podcast is, how is it playing into voters' calculus? And is it shaping who they'll select for president in 2024? This is a really big part of Biden's argument for re-election, and it seems to be less a part of Harris'. So has that changed how voters are thinking about this issue at all? And more broadly, how are they thinking about it?

I mean, I think this issue is basically baked in. I don't know that we are going to get news that's really new on this. We may get news that makes headlines, that has new details, but I don't think it's fundamentally going to be changing the shape of the story. So to the extent that this is an issue impacting voters' choices in November, it probably has already had all the impact it's going to have.

Well, then I guess I have maybe a more specific question here, right? Harris campaigned with Liz Cheney on Friday in Ripon, Wisconsin, which is where the modern GOP was founded. And Cheney spoke pointedly about January 6th. And she makes clear, you know, she was not a never Trumper. She was in Republican leadership in the House throughout Trump's presidency and didn't make a point of butting heads with him.

So how big of a part of the electorate is this that, you know, they could get on board or at least make do with most of Trump's behavior, but that his behavior around the 2020 election was simply a bridge too far?

I think it's a pretty small part. So I pulled up some data from YouGov in August, and they asked about all the cases facing Trump, the Georgia case, the January 6th case, classified documents case, and the Stormy Daniels case. And if you look at self-identified Republicans, right, because that's who Liz Cheney is really trying to message to.

On all of those cases, you have something like 20% of self-identified Republicans saying that they're very serious or somewhat serious. You have something around for all four cases, something close to 10% of self-identified Republicans saying Trump should be convicted. But then when you look at

polls of the actual horse race matchup of Harris versus Trump, you don't get anywhere near 10 percent of Republicans saying that they're not going to vote for Trump. So even Republicans that have concerns about these things, they're still most of them are still planning on voting for Donald Trump. There's really just a very small segment of

of those self-identified Republicans who have concerns and that those concerns are going to move them off of voting for Trump, right? We have something like 4% or so of Republicans on average saying that they're planning to vote for Kamala Harris. Yeah, if you want to know how many people like that there are, we do have a good indication of how many there are in Wyoming because she's no longer a House representative from Wyoming.

The New York Times, Sienna pollsters put together all of the not firmly decided voters from their most recent battleground state surveys. And they asked them all to describe in their own words, what do they have concerns about when it comes to Trump? What do they have concerns about when it comes to Harris? The biggest concern about either candidate that's mentioned is

about Trump, over 30 percent of those on just not necessarily undecided, but not firmly decided voters, over 30 percent say their biggest concern about Trump is his personality and behavior. The threat to democracy is less than 10 percent. Right. That's just not the thing that bothers them or that they have concerns about when it comes to voting for Donald Trump. Which I think is maybe a little hard to disentangle the personality and behavior of Donald Trump

January 6th versus the threat to democracy. I mean, you could think that what Trump did on January 6th and in the run up to it were wrong, but that ultimately our institutions held. And so it wasn't it's not a threat to democracy, but it's bad behavior. Nonetheless, I don't know. I'm just spitballing here. But it seems hard to exactly parse what voters might be thinking about there. No, totally. And this is an open ended question. So they're having to bucket it up however they bucket it up. Yeah. And I guess I wasn't thinking that this

that this unsealing was going to change people's minds. I'm more curious about who the voter is who's already decided to part ways with Trump because of this factor. And maybe they decided to part ways with Trump four years, not exactly four years ago, but three and a half years ago and not today. I mean, that's certainly Liz Cheney's circumstance or Adam Kinzinger's circumstance.

And I'm just curious how large a part of that electorate is. Maybe they no longer identify as Republican even. Maybe they identify as independent. Maybe even some of them identify as Democrats. This is a great segue because who that voter is, is the white voter that's switching from Trump to Harris that we're going to talk about in the next segment. All right. Well, then you did the segue for me. But first, a break.

Today's podcast is brought to you by Oracle Cloud Infrastructure, or OCI. AI might be the most important new computer technology ever. It's storming every industry and literally billions of dollars are being invested. So buckle up. The problem is that AI needs a lot of speed and processing power. So how do you compete with costs spiraling out of control? It's time to upgrade to the next generation of the cloud, Oracle Cloud Infrastructure, or OCI.

OCI is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds. It offers one consistent price instead of variable regional pricing. And of course, nobody does data better than Oracle. So now you can train your AI models at twice the speed and less than half the cost of other clouds.

If you want to do more and spend less, like companies Uber, 8x8, and Databricks Mosaic, take a free test drive of OCI at oracle.com slash 538. That's oracle.com slash 538. The numbers, not the letters. oracle.com slash 538.

Today's podcast is brought to you by SimpliSafe. SimpliSafe has been named best home security system by US News and World Report for five years running and offers the best customer service in home security according to Newsweek. It's easy to install and activate your SimpliSafe system in less than an hour or choose professional installation to have a pro do it for you.

With Fast Protect monitoring and Live Guard protection, SimpliSafe agents can act within five seconds of receiving your alarm and can even see and speak to intruders to stop them in their tracks. You'll never be locked into a long-term contract, so you can cancel any time.

Pricing is transparent and affordable at less than a dollar a day with no hidden fees ever. Protect your home with 50% off a new SimpliSafe system, plus a free indoor security camera when you sign up for fast protect monitoring. Just visit simplisafe.com slash 538. That's SimpliSafe, S-I-M-P-L-I, safe.com slash 538. The numbers, not the letters. There's no safe like SimpliSafe.

In a polling environment that's been relatively stable, it seems like some of the biggest stories of the 2024 election may actually be in the crosstabs. So throughout the year, the crosstabs have continued to show that Americans seem to be realigning. Black and Hispanic voters increasingly supporting Trump, Harris improving over Biden's support with white voters, young voters warming to Trump, and seniors potentially even giving the majority of their support to Harris.

It wouldn't have surprised us if these trends faded as we got closer to election day, but that doesn't seem to be what's happening. By and large, these demographic shifts still show up in the data. So what might that mean for how this election will play out? And do we buy what these crosstabs are actually showing? So Mary, you dove in to the crosstabs headfirst. You looked at national polls from the past month and averaged all those crosstabs. I sort of gave the big picture, but what did you find?

First of all, we see a really, really stable race. There's been very little change from the last time we did this in August. After the first month of the Harris campaign, we took a look and if you compare what we're seeing in the crosstabs now to then, very little has changed. However, that is kind of interesting, right? Because that may mean that the trends we started to observe in August are real, right? This is something that's really happening.

The trends that I think are most interesting here is you see Harris improving upon how Biden performed in 2020 with a couple segments of the electorate that are really important, in part just because they're reliable voters and there are a lot of them. I'm specifically thinking of white voters. Over half of the United States electorate is white. Well, 70 percent in this election will be white. So like.

To improve even just a couple of points with white voters has an enormous impact, much more than slipping even by 20% on the margin with black voters. It's suburban and white college educated voters, again, moving closer to Harris than they were to Biden in 2020. And as you mentioned, older voters.

To the points that we were talking about before the break, this, I think, is really the kind of voters we were discussing that may have been moved by Trump's behavior around January 6th. All of those sort of distasteful elements of the end of the Trump presidency, these may be the voters that are the ones saying January 6th is the last straw. I used to vote for Republicans, but I'm not voting for Donald Trump this time around. And what are we seeing on the

the flip side where Harris is losing ground to Trump. And I should say some of the voters that you've described may also be moved by an issue like abortion or something like that could have reliably voted for Republicans when the question wasn't live on the ballot or live in government and the likes. Across the board. And even, you know, we talk about crosstabs can be

little bit noisy. The sample sizes are really small. But even if you look at polls focused on certain subgroups like Black voters and Hispanic voters, Democrats are seeing significant attrition among those groups. So in the national crosstabs, Harris is getting about 82 percent of the two-party vote share among Black voters. That's compared to 92 percent in the 2020 exit polls.

So that's a 10 point drop for Harris, 20 points on the margin. And that's been fairly consistent across all kinds of surveys. Some drop there. Something similar is happening with Hispanic voters, where in the national crosstab, she's only seeing a 58 percent support among Hispanic voters compared to 2020. That's a pretty significant drop also. So

this sort of interesting racial realignment thing is starting to emerge where she's getting a dribble of white voters coming into the Democratic camp and a pretty significant loss among voters of color.

Anyone who spent time on election Twitter or just read some of the near-to-your parts of election coverage online knows that there's debate about the degree to which this realignment is actually happening. You know, it is the case at times that we see dramatic shifts in the crosstabs that turn out to not happen on election day. You know, the margins of error are larger when we're talking about the crosstabs, and some of these parts of the electorate can be particularly difficult to pull. So,

How skeptical should we be that this realignment is happening? You know, if you look at polls like polls specifically among black voters or specifically among Hispanic voters, you see less attrition for Democrats with those groups in those surveys. But you do still see some drop off. If you look at, for example, there was a Howard University poll that was just looking at black voters in the key swing states.

If you look at the two party vote share, so basically throwing out the undecideds and trying to get a sense of just like for people who are going to vote, where are they at? They have 87 percent support among black voters in that survey, which is still less than 92 percent that you had in the 2020 exit polling survey.

But it's a little better than the 82% we see in the national crosstabs. So I think the truth is probably somewhere in between. I would expect that we will see some level of attrition among voters of color. I think probably will be more dramatic among Hispanic voters based on, you know, surveys we've had.

from BSP Research, which focuses on Hispanic voters, some surveys that were sponsored by Telemundo. They all show more drop off among Hispanic voters than we see among Black voters in surveys focused on Black voters. So that could have a significant impact, particularly thinking about like Arizona and Nevada are sort of Western Sunbelt states.

I'm going to pick up on one thing, and that is this question of how accurate the crosstab averages are. First off, we have a lot of crosstabs, so there's virtually no sampling error in these crosstab averages. There is a question of whether or not there's non-sampling error, whether or not the people we're talking to are representative crosstabs.

of the groups, even if we had like talks to infinite of them, we hear as pollsters, not necessarily 538. But in 2020, the crosstab averages were good with Hispanic and Latinos, with black voters and with young voters, and bad with non-college educated white voters and with seniors. The polls overestimating a shift towards Democrats among the latter group. I mean,

I mean, it's one of the cases where if you looked at the crosstab averages, you probably would have had a better idea of the ultimate election outcome than the top lines themselves. And this was because pollsters had the wrong mix of the electorate demographically, with some of them not waiting to the right variables to get education correct and having too many white-educated people in it. It's not obvious to me that we're seeing anything here that should be a red flag or that...

you know, that is conflicting other data. So I pretty much believe it. The other point I would make is that when you look at particularly voters of color, you tend to see more undecided voters in polls at this point. And because those groups have been historically Democratic-leaning, you might expect that this sort of drop-off for Democrats among voters of colors might lessen a little bit as those undecided voters make their selections. Yeah, there's an interesting aspect here, too, which is, Elliot, you're saying that

that in past elections polls overestimated white support for Democrats. And

And so you might say, well, what is the oversample of white voters show? Are there pollsters out there really focusing on non-college educated white voters or white voters in general? There aren't. But I was thinking of like what could be a proxy for an oversample of white voters. And maybe the closest you'll get is Ann Seltzer's Iowa poll, right? I mean, Iowa is almost 90% white. And if you look at the most recent Iowa poll, you see a significant shift or notable shift

towards Harris, which is that it's a four point race. Trump leads by four percentage points in Iowa compared to winning that state by eight percentage points, seven or eight percentage points in 2020. So you could make the argument that an oversample of white voters, quote unquote, you know, not a real oversample, but is also showing some shifts.

Do you buy that argument? I mean, I think that's an interesting approach. If you think about Iowa, it's not just white voters. It's also significantly rural. And if we look at the national crosstabs, we don't see as much movement with the rural community. And the movement among whites we tend to see is generally among college educated white voters. So I don't know. It's an interesting approach. But I think there's like a demographic mix there that's not quite representative of the population of white Americans generally.

All right, so there are some caveats here, but let's just pretend for a moment that this realignment as we see it in the crosstabs right now ends up materializing on Election Day. How would this shape the Electoral College system?

a sort of gain for Democrats amongst white voters and a gain for Republicans amongst voters of color. Well, FiveThirtyEight has a fun tool for playing with this. We published earlier this year. It's called the Swing-O-Matic. And you can go to the site and put in your margins with each demographic group and see what happens.

So I put in the margins we're currently seeing in the national polling for white, black and Hispanic voters. Those are the only three things I changed. Everything else stayed exactly the same. And the outcome is a 270 to 68 race, which is perhaps one of the most terrifying things I've had to write about. So if the polls were exactly accurate on the racial demographics and nothing else changed from the last election, we're in for a nail biter.

Okay, so Elliot, this makes me think of a piece that you recently wrote, which is that this could be the closest election since 1876 when Rutherford B. Hayes beat Samuel J. Tilden by one electoral vote.

So on one hand, we could be looking at an incredibly close election. But at the same time, we know that polls are off on average in a presidential election by about four percentage points. And because things are so close today, if you move them in either direction by four percentage points, you're borderline looking at a landslide for either Harris or Trump.

So with all this talk of a close election, are we underrating the possibility of a landslide, Elliot? I mean, I wouldn't call it a landslide. An election night winner, let's say, because for our current political moment, knowing who won on election night almost feels like it must have been a landslide. Yes. In that definition, yeah, I think we are underrating this.

Specifically, yeah, if you swing all of the states that are within four and a half points today toward Trump, he would win with 312 electoral votes. If you do the same thing for Harris, then she would win, depending on whether or not you believe the polls in Texas, 319 or 360, if you believe Texas is as close as the polls say. I'm from Texas, so I'm just going to use the conditional. Okay.

That would be an entirely different electorate than the one that the polls are picking up on today, essentially. By like two percentage points on all these crosstabs, just shift it four points on margin, and you get an entirely different discourse about how this election is going. You'd get narratives about

How the country decided that they were done with Trumpism and his bad personality or blah, blah, blah. Or that they placed their emphasis on inflation and economic growth above abortion or democracy. You're going to get something on either side there. I think the difference in our potential narratives on election night are important for us to think about because we haven't observed that yet.

So to be clear, are we still saying that the likeliest outcome is a very close race? If you believe that there's going to be an average four-point error, it's going to go toward somebody, and someone's going to win 300 electoral votes. If you believe that pollsters have fixed their problems from the past and you should expect no error on average, then you've got a close election in a different definition.

What an interesting, interesting dynamic in 2024. But speaking of polls and how to, you know, fix potential problems in the polls, one group has come up with quite an unconventional way of doing it, which we're going to look at, but first, a break. Today's podcast is brought to you by Shopify. Ready to make the smartest choice for your business? Say hello to Shopify, the global commerce platform that makes selling a breeze.

Whether you're starting your online shop, opening your first physical store, or hitting a million orders, Shopify is your growth partner. Sell everywhere with Shopify's all-in-one e-commerce platform and in-person POS system. Turn browsers into buyers with Shopify's best converting checkout, 36% better than other platforms.

effortlessly sell more with Shopify Magic, your AI-powered all-star. Did you know Shopify powers 10% of all e-commerce in the U.S. and supports global brands like Allbirds, Rothy's, and Brooklinen? Join millions of successful entrepreneurs across 175 countries, backed by Shopify's extensive support and help resources.

Because businesses that grow, grow with Shopify. Start your success story today. Sign up for a $1 per month trial period at shopify.com slash 538. The number is not the letters. Shopify.com slash 538.

We often say that polling, even if it is error-prone, is still very important because in a democracy, it matters what voters think. Well, in today's good or bad use of polling, the pollsters are taking the voters, the Americans, out of the equation altogether. The news site Semaphore recently profiled Aru, I think I'm saying that correctly, an experimental polling company founded by two 19-year-old college dropouts.

What makes them different from any other polling company is that they don't actually poll anyone. Instead, they program thousands of AI chatbot agents to respond like the people in a given area. They input census data, personality traits like aspirations and family relationships, and a steady diet of media meant to copy what folks consume online. These quote-unquote polls take about a minute to conduct and cost about a tenth...

of a normal poll, human poll. According to the founders, they've already been hired to conduct polls for political campaigns, think tanks, and super PACs. Notably, they apparently predicted Representative Jamal Bowman's 17-point loss to George Latimer earlier in this year in the primary by within 371 votes. So, Elliot, you have been champing at the bit for a minute to talk about this use of polling, although I'm not sure I should call it polling. I'm

Either way, I'll ask the usual question. Is this a good or bad use of polling, Elliot? Bad. Hey, Galen, you know what else also predicted Jamal Bowman losing to George Latimer? Polls. Actual interviews with people. You know, you could actually ask people how they think instead of like...

running interviews that are from polls through some sort of large language model to interview archetypes of people. You spend a lot less money on an LLM and maybe a little bit more money on a poll of a congressional district and get the same result and actually be doing the thing that pollsters are trying to do, which is emulate democracy and stuff. And I wouldn't even call it a poll. We've said poll a lot here, but what we're really looking at is some

sort of weird AI algorithm that is emulating the survey research process. But there's no sampling involved. There's no, you know, modeling in the way that a pollster or survey researcher would understand it. So it's a bad use of whatever it is, and it's not even polling.

So let me ask, because I think the main argument we've made here is that it's not, in a sense, democratic or righteous in the sense that if we care what people think, this is not actually asking people what they think.

But from the perspective of getting an accurate sense of where the mood of the country is in the abstract, could you say then that it's a good use of large language models? No, because we don't know that it's accurate. We know that maybe they emulated something that they could have distilled into a point estimate for an election campaign and say, oh, we did this one congressional district correctly. But we don't know...

empirically how that's going to translate to future performance, if you're thinking about this from a model-driven thing, point of view. And we know that polls are accurate to some theoretical minimum based on sample size and difficulties representing people that are hard to sample because there's a theory of measurement behind it, which is that if you talk to people and those people are representative,

and you talk to enough of them, then you will approximate the outcome within some margin of total error. Central limit theorem. Central limit theorem plus extra non-sampling bias in the survey. So there's no sense that what they're doing is legitimate, period.

There's no, they're not using any scientific method. You're saying they're just like f***ing around and finding out. Yeah, I think it's FAFO polling. Maybe they should rename themselves from AARU to FAFO. It's just not, they're just like,

This is the type of thing that you would do that would be like, that you could like test for years and maybe come up with some sort of addition to your like package as a strategy firm. It might say, oh, here's, if you're consuming this media diet, here's what you might think. If you're consuming this media diet, here's what you might think. And then you can, you know, think about what voters might go into different buckets. And then like that could be a jumping off point. And actually that is the way that they're doing.

Doing this using LLMs is pretty interesting on that axis, but this is not election forecasting. It's not surveying. It is LLM AI bulls**t. And I'm afraid for the company that it's going to backfire. And then I'm also afraid for us as democracy that we would even believe that we should not be doing polling and instead be doing this. So that's my not strong at all opinion about this company. Yeah.

Mary, you've been awfully quiet over there. I was just going to let Elliot say his piece. I have so many questions about what on earth is going on over there. First of all, they said that they predicted that one primary by 371 votes. What does that mean? Are they modeling turnout? Usually we, when we look at a poll, we're thinking about like percentage point margins. So like, I don't even know what they're talking about when they say 371 votes. That's not how we would present polling information. It doesn't make any sense. And I was just like...

like deeply concerned that like they're in this huge entrance into the world with like a case study of one. If we were rating a pollster in our pollster ratings with only one poll, we wouldn't be able to give a confident rating, right? Like it's just, it's not enough information. And they probably showed us whatever they thought was their best result.

to get people interested in this. The thing that really irked me reading that semaphore piece is they, because these are like 19 year old kids, right? So they made some very confident assertion that we'll never be doing polls again. We're always going to do AI modeling of the election from this point forward. It's so much cheaper.

You might think that if you have no conceptualization of the value of democracy. And in order to do what they're doing in the first place, they have to have polling data. They're relying on the American Community Survey, which is a poll conducted by the United States Census Bureau every year. They can't do this without polling. The single most expensive probably data set in America, right? I mean, this is a poll where the sample size is, I mean, first of all,

It starts with the census, which is not even a poll because the goal is to talk to every single person. So it's not a representative survey. It's literally just counting every single American. Then the American Community Survey talks to millions and millions of Americans in order to update that information. And so it's not starting from a place of you don't need polls. It's starting from a place of the most expensive data in America. In America.

I just feel like they're making a lot of assumptions about how people consume media. You know, one thing I think about all the time when we look at how people read social media in particular is this Pew study from a few years ago that found that like only 10% of people that use social media post. Like if you think about Twitter, 90% of the users of Twitter are silent people that are just reading the information. So I guess it's sort of hard for me to imagine how do you interpret how those silent people consume media?

what they are seeing on social media. The LLM is going to make a guess and it's going to make a very confident guess because that's what LLMs do. But I just don't have any reason to believe that it's going to be good at understanding how human people like internalize the media that they see.

All right. Well, clearly, this has been our first good or bad use of polling war against artificial intelligence. Listeners, you will have to decide for yourself who won the first battle in what will be a long war. But we're going to leave it there for today. I do not welcome our new AI overlords. No, thank you. Thank you, Elliot and Mary. Thanks. Thanks, Galen.

My name is Galen Druk. Our producers are Shane McKeon and Cameron Chertavian, and our intern is Jayla Everett. You can get in touch by emailing us at podcasts at FiveThirtyEight.com. You can also, of course, tweet us with any questions or comments. If you're a fan of the show, leave us a rating or review in the Apple Podcast Store or tell someone about us. Thanks for listening, and we will see you soon.