Chapters
Shownotes Transcript
See CapitalOne.com slash bank Capital One N.A. member FDIC.
Freakonomics Radio is sponsored by Mint Mobile. Ditch overpriced wireless by switching to Mint Mobile and get three months of premium wireless service for $15 a month. All plans come with high-speed data and unlimited talk and text delivered on the nation's largest 5G network. You can use your own phone, your own phone number, and all your existing contacts. To get this new customer offer and your new three-month premium wireless plan for just $15 a month, go to mintmobile.com slash 5G.
That's mintmobile.com slash freak. Cut your wireless bill to 15 bucks a month at mintmobile.com slash freak. $45 upfront payment required, equivalent to $15 per month. New customers on first three-month plan only. Speeds slower above 40 gigabytes on unlimited plan. Additional taxes, fees, and restrictions apply. See Mint Mobile for details.
Today on Freakonomics Radio, a very special episode, a conversation about the late Daniel Kahneman, whose insights into human behavior have been threaded through this show for years. Ideas like confirmation bias and loss aversion and the planning fallacy.
During this conversation, we also learn about a research paradigm that Kahneman embraced called adversarial collaboration, which means working shoulder to shoulder with your rivals. He felt that this is the right way to do science. Kahneman was a phenomenally influential psychologist who won a Nobel Prize in economics, Ray
This is Freakonomics Radio, the podcast that explores the hidden side of everything with your host, Stephen Dubner.
Last month, in a sunlit auditorium overlooking the Chicago River, there was a gathering of psychologists, economists, and other social scientists. This was the Behavioral Decision Research and Management Conference. The keynote event was supposed to be a conversation with Danny Kahneman, facilitated by Richard Thaler, his longtime friend and collaborator.
Thaler is the University of Chicago economist who helped turn Kahneman's insights into the field now known as behavioral economics. But when Kahneman died in March at age 90, Thaler came up with a new plan for the conference. Now it would pay tribute to Danny Kahneman. Freakonomics Radio was lucky enough to be asked along to moderate a couple of panel discussions about his life and work.
The episode you're about to hear is a condensed version of those conversations. This all took place at the downtown outpost of the University of Chicago's business school in front of a couple hundred attendees.
Some of the panelists had known Danny Kahneman for many decades. For instance, the psychologist Maya Bar-Hillel. Her father was a philosophy professor at Hebrew University in Jerusalem, where Kahneman got his undergraduate degree. My father taught Danny and gave him a lot of grief. But my father apparently gave just about everybody a lot of grief.
He was a tough-minded philosopher. And when Kahneman became a professor, one of his students was Maya Bar-Hilal, from generation to generation. I met Danny in the first week of my first year at the Hebrew University. He gave the introductory statistics course. He looked at us and he pronounced right away, you are the creme de la creme.
He said it in French, and we were. Kahneman had grown up in France during the Nazi occupation. He survived, barely, and lived for many years in Israel before coming to the U.S. to get his Ph.D. at UC Berkeley.
His research partner, Amos Tversky, was another Israeli psychology professor who moved to the States. Tversky died young in 1996, too early to share in what would have surely been a joint Nobel Prize. Tversky was regarded as perhaps even more brilliant than Kahneman. The two of them published many papers on judgment and decision-making, but not just in psychology journals.
It always struck me just in the story of how Kahneman and Tversky research was taken hold of by Thaler and others and turned into what we now call behavioral economics, that it was very, very important that these were two psychologists who were also very mathematically fluent.
If that hadn't been the case and if the publication hadn't been in, I guess, econometrica and so on, was there a pretty good possibility for a counterfactual where all that research might have stayed within the realm of psychology and never trickled over and we might not have what we think of as behavioral economics? Yeah.
It was not an accident. It's not how fortunate that they went to Econometrica. They realized that their work was attended to primarily by psychologists. And in fact, they both considered themselves all their lives as psychologists.
psychologists, but they also realized that their research was perhaps more important outside of psychology. So the decision to publish their paper in Econometrica was a deliberate move. It was a strategic move to get the attention. They believed that that was the ticket and without the ticket, they would not have been playing in that field.
A lot of the early Kahneman-Tversky work centered around an observation that may seem obvious in retrospect, but at the time had not been explored with much rigor. The idea was that we are all constantly making decisions, personal, professional, political decisions. And then later, if we ask ourselves why we made a given decision, which, by the way, we usually don't ask, we
We might tell ourselves a story about making the decision, but these stories are often not quite true. Why? One reason is that we want to appear to others and maybe even ourselves as smarter than we are. Here is Richard Thaler. I met Danny and Amos in 1977, and it was a transformative year for me. I decided to change jobs and took a job at Cornell.
and decided to offer a course in the thing I was now fascinated by and called it behavioral decision theory, which is kind of what the name of this field used to be. I got about eight students. So I had to think of something to do to increase the enrollment in the class. And so I changed the name to managerial decision-making. 50 students show up.
I began the class by asking, how many of you signed up for this class because of the name? No one raised their hand. I said, well, actually, nearly all of you. That's one illustration of what Danny's talking about. No one thinks they would be stupid enough.
to sign up for a class based on the name. I believe it was in Thinking Fast and Slow where Danny wrote, not only are we blind, but we are blind to our blindness. Yeah, and you know, maybe one of the many secrets of the Kahneman-Tversky collaboration was they were not blind to those. And what they had was...
mistake detection facility. They had some radar where they could anticipate what the mistake is. And because they had this mistake detection facility somewhere, they were able to figure out things. And then they were pretty good at predicting what people other than Amos and Danny would do.
To me, the big point, the aha point that I got from reading their judgment papers was the idea of predictable bias. May I add something to that? Yes, please. Not only predictable. I think now I'm going to say something that is perhaps I can't say that I heard them say it, but I hope they would both agree.
The errors are not just predictable. The errors are smart. Our stupid mistakes are evidence of the wonderful human mind. That's how normal cognition functions. And normal cognition is amazing.
I love that. It does make me want to ask any, all of you, a question that's fairly heretical, which is, you know, as a layperson, I read these findings. They're attractive. They're believable. They're identifiable. But the thing that I always struggle with or would like to understand better is how you all can feel so confident that they're generalizable. Yeah.
There must be people in the world who are not susceptible to loss aversion or recency bias, or many, many of them. So really the question is, how much variance is there, and how do you measure the variance? You know, Amos used to have a joke that there were species that didn't exhibit loss aversion, and they're now extinct. LAUGHTER
Think about what Maya Bar-Hillel was saying there about the smart errors we all make. It would be easy to overlook the baseline insight that Danny Kahneman offered, that people make thinking mistakes all the time. Now, most of us, upon hearing this, might say, no kidding. Our species is highly fallible? Who doesn't know that?
But it is our fallibility, Kahneman realized, that makes us interesting and worthy of inspection. He accepted that humans are capable of wonderful things as well as terrible ones, but that we are overconfident in our abilities, that we often have poor self-control, and
and that we employ an arsenal of mental shortcuts or heuristics to make decisions or reach judgments that often turn out poorly. And even when presented with evidence of our mistakes, we usually fail to change our minds.
Here is Eldar Shafir, who runs the Kahneman-Triesman Center for Behavioral Science and Public Policy at Princeton. He and Kahneman used to co-teach a class. One lecture that I love that Danny used to give in our course, we would talk about people's not good sense about conditional probabilities. And then we had a clip of the O.J. Simpson trial, where Dershowitz explains to the jury that
that the probability that a beaten woman is going to be killed by her beating partner is extremely low, which is true. However, we're not predicting the chance that Nicole Simpson will be killed. She has been killed. The probability that a beaten woman who has been killed was killed by her partner is immensely high. That nuance of not being sensitive to these conditionals has major implications.
He was so genuinely curious and intellectually alive. He presented the same way to the Swedish monarchy and to an undergraduate. He just wanted to think about it. He wanted to struggle with the question. He wanted to listen and think of the best theory. That was what's so impressive. When we taught together, we took turns lecturing.
Here was Danny, I mean, before and after the Nobel, and every class he would come and sit down with the students and listen. He could easily not have shown up. It's other people's lecturing. He was always there, asking questions, answering questions, devoted to understanding, always had a new thought about something that we have done on slides for three years in a row.
Many things that were written about Kahneman, I'm always amused by the fact that they say he was aware of these quirks and fallacies, but he was aware that he has them too. And that sort of implies that since he was aware, it wasn't as big a deal. There's a couple of recorded interviews with Danny during COVID where he predicts that COVID means life imprisonment for the more advanced in age.
And, you know, he was wrong. He lived a couple of wonderful years after COVID. There was a sense in which he really sank deep into the same error that he was able to predict and acknowledge. The other amazing thing about Danny is when he won the Nobel Prize, you have to give a lecture. And he gave a lecture and decided he was going to give a new interpretation of
of all of his work with Amos. Using this two-system approach, I kept saying, Danny, you have two months to do this. Why are you starting over? They gave you a Nobel Prize for what you did. You don't need a new reinterpretation. But that lecture in Stockholm...
was the beginning of what led to thinking fast and slow because he was reinterpreting all the heuristics and biases stuff
in the lens of System 1 and System 2, which is your immediate reaction versus when you sit down and think about it. What this story suggests to me, but tell me if I'm wrong, is that he understood that having an audience like the Nobel Committee audience and the king or queen or whatever of Sweden...
and that his voice was now amplified, that he wanted to take the work that he'd done and make it accessible, make it known to people who have some levers of power. He wanted to exploit a good opportunity. I don't like the word exploit. Exploit for pro-social reasons, a good opportunity. Thank you. Thank you. Yeah, I think he had an audience and wanted to get it out there, but
It was torture for him. Absolute torture. I mean, during the writing of Thinking Fast and Slow. Which was a long gestation. A long gestation. And he decided absolutely positively that he was quitting.
Two dozen times? You know, Danny, in some sense, really was a bon vivant. I mean, he enjoyed life. He loved a good restaurant. He loved a good vacation. And I think he loved getting the prize. I don't think he changed. I mean, yes, the world was paying more attention to him, but he was always Danny. He changed. He was happier. Okay. Now, I would like to say... If Maya says that, I accept. I would like to say in what way?
He relaxed. That was relaxed? Yes. Yes.
What do you mean by that, Maya? He had made it. There was a common misperception that the Dvorsky-Kahneman collaboration was not symmetrical. The world seemed to think that Amos was the lead and Danny was an outside visitor. It was a symmetrical and equal collaboration. But the
The world didn't know it. It was very important to Danny to be able to do good work after Amos dies so that people won't continue with this error that he was less than equal. So the prize told the world, this man is noble worthy and he did excellent work because he had an excellent mind. But also it was...
important for him that Kahneman not die the name together with Dvorsky. And it didn't. As I said earlier, the main ideas of Danny Kahneman have been threaded through the fabric of Freakonomics Radio over the years. Episode 323, for instance, which was about the planning fallacy. It's called Here's Why Your Projects Are Always Late and What to Do About It.
Then there's episode 271, The Men Who Started a Thinking Revolution. That was an interview with Michael Lewis about the book he wrote on the Kahneman-Tversky partnership. Lewis's book is called The Undoing Project, A Friendship That Changed Our Minds. So, yes, you may be familiar with Kahneman's greatest hits, but one of the conversations we had at this conference went into an area I knew very little about, and I bet you don't either.
That's coming up after the break. I'm Stephen Dubner, and this is Freakonomics Radio.
Thank you.
LinkedIn does all that while making the process easy and intuitive. Hiring is easy when you have that many quality candidates. So easy, in fact, that 86% of small businesses get a qualified candidate within 24 hours. Post your job for free at linkedin.com slash freak. That's linkedin.com slash freak to post your job for free. Terms and conditions apply.
T-Mobile has home internet on America's largest 5G network for 50 bucks per month. It's how I stream the game. Oh, yeah, that one's out there.
It's how I knock out the shopping list. Hey, hon, the delivery's here. Oh, cool. That was easy. It's how I level up. Yes, double points. Get T-Mobile 5G home internet for only 50 bucks per month with auto pay and any voice line. And with price lock guarantee, your last month of service is on us if we ever raise your internet rate. Plus, there are no exploding bills or annual contracts.
T-Mobile, it's how you internet. Check availability today at T-Mobile.com slash home internet. Price lock exclusions like taxes and fees apply. See guaranteed details at T-Mobile.com. Price after $10 monthly bill credit while you maintain a nationally available postpaid voice line. Qualifying credit required. Regulatory fees included for qualified accounts. Plus $5 per month without auto pay. Debit or bank account required. Good morning. Good morning. Good morning.
We have a session this morning, as you know, on adversarial collaboration. Day two of the conference with a panel devoted to a different way of doing business in the behavioral sciences. Let's begin with a short clip from the late Danny Kahneman. This was recorded in 2022.
Controversy is a terrible way to advance science. It is normally conducted as a contest in which the aim is to embarrass. The feature that makes most critiques intellectually useless is a focus on the weakest argument of the adversary. It is common for critics to include a summary caricature of the target position, refute the weakest argument in that caricature, and declare the total destruction of the adversary's position.
It's rare for anyone to concede anything. Doing angry science is a demeaning experience. Doing angry science, Kahneman came to believe, was a terrible thing. He knew this firsthand. Before the accolades, the prizes, the acceptance...
Kahneman and Tversky came under a great deal of criticism. Kurt Gigerenzo published his first critique of our work 37 years ago, and he's still not done with me. And here again is Richard Thaler. I've certainly had my share of angry science exchanges. They're no fun, and there's never any light, only heat. And that's why Kahneman came up with a different model—
what he called an adversarial collaboration. He felt that this is the right way to do science. All of the collaborations shed more light than heat. The typical debates in academia are exactly the opposite.
So I think Danny has a good point. Kahneman was trying to solve a fundamental problem in social science research. It's natural for scientists to disagree with one another, but there's no clear mechanism for resolving those disagreements. So what if researchers who disagree work together in good faith to resolve the issue? And what if they take on a neutral third party to serve as an arbiter?
That is what Kahneman called an adversarial collaboration. He participated in a variety of them over the years. The conversation you are about to hear includes some of the key players. Let's start with introductions. I'm Tom Gilovich. I'm a social psychologist at Cornell University. I'm Barbara Millers from the University of Pennsylvania at
And I was fortunate to be the arbiter in two adversarial collaborations with Danny 20 years apart. I'm Matt Killingsworth. I'm also at the University of Pennsylvania. And I study human happiness and well-being, like what makes life worth living and how do we collect data to try to understand what makes life better? My name is Shane Frederick. I am a professor at the Yale School of Management.
Tom Gilovich, let's start with you. Let's hear basically the story of the adversarial collaboration with Danny. It began quite early in your career. Give us the whole story.
I had published with Vicki Medvek a chapter in a book on counterfactual thinking. When I wrote the chapter, I don't know if I was aware that Danny was going to write the wrap-up chapter where he comments on all of the other ones. So I don't know if his voice was in my head as we're writing the chapter. What's the chapter about? What's it cover? A lot of it covers the subject of regret, what we regret most in life. Amos and Danny had published a study showing
you regret things of action more than inaction. The example they used is imagine you own stock in one company, you think, okay, that's run the course, you switch to another and it tanks, you lose a certain amount of money versus you're thinking about buying this stock, you decide not to, it takes off and you lose the same amount of money, which would you regret more? And almost everyone anticipates that you would regret the action the most.
Nonetheless, if you ask people, what are the biggest regrets in your life? What dominates are things they didn't do. And so a lot of the chapter was, how do you reconcile those two things? In approaching this riddle or puzzle in your mind, how did you measure what was your actual research? Our research was what's responsible for that difference? How do things change over time? Precisely because your regrets of action really hurt
you do things about them. Sometimes you change your life accordingly, or you certainly engage in lots of psychological work to achieve some level of peace. Because the inaction regrets gnaw at you less powerfully, you sort of leave them alone. The fun of that research was tracking down all of the different psychological processes that
make those regrets of inaction hang around or in some cases even get more intense whereas the regrets of action dissipate. How do you do that though? Is it empirical? Is it theoretical? It's empirical. For example, one of the mechanisms that we thought was especially interesting is when you don't do something that you later on end up regretting
There are reasons for that. Like, I just don't have the bandwidth right now to take this project on, etc. Those are compelling reasons in the moment for why you're not doing this thing. You move along in time, you look back and you think,
Wait a minute. I could have done that. So we would do studies where we'd ask Cornell current students, recent alums, or much older alums, imagine there's a class that you always wanted to take but maybe you were a little afraid to. Suppose we added that to your curriculum this semester. The students are like, oh, I can barely hang on right now.
You add that, it's a catastrophe. You ask the people who graduated many years ago, they think, oh, that's a piece of cake. It wouldn't have interfered with my GPA, my social life, the amount of sleep I get, et cetera. Okay, so you write up the findings. At what point then and in what way do you hear from Danny? It's sort of a highlight of my career. I was on sabbatical, check in your voice messages back at Cornell, and I get this message that
Tom, this is Danny Kahneman. I just read your paper with MedVic. It's brilliant. It made my day just calling to say thank you. What a piece of work. And then I stupidly press star delete instead of saving that message forever to play to my grandkids. So I'm just on a high for a while.
Two weeks later, there's another phone call from Danny Kahneman, this time live. Oh, great, my new best friend is going to tell me about... Tom, I've been thinking more about your chapter. It's all wrong. I've heard you're planning to publish this. You can't do that. If you do, people will go after you. And if no one does, I'm going to go after you. That is...
What happened to brilliant? Did he, in that conversation, give you his argument for why he thought now you were wrong? Yes. He said that there really isn't a change in the intensity of either action or inaction regrets. There's just a substitution of the kinds of things that you're regretting. And the regrets of inaction really aren't serious regrets. You might not even call them regrets. They're more wistful, like,
Oh, I wish I had learned to speak Esperanto. Said no one ever. All right. So what happens next? Well, lots of panic for a few days. Vicki Medvik and I, well, what do we do?
It can't be worse than this. And I think, I forget what, there had been an example of an adversarial collaboration. I think it was Vicky's idea. Hey, we should negotiate with him. Let's see if we can do an adversarial collaboration. So we pitched it and he was very receptive. This was in what year, Tom? Chapter came out in 95. So it was a little before that. So this was pre-Zoom, certainly, early internet.
How are you communicating and how did you negotiate the negotiation and how did the actual negotiation happen? I'm glad you asked that question because it allows me to bring up a point about Danny that there's been all this talk about his many qualities and they've all been well said. One thing kind of gets left out. I mean, to be at that level of success is
You have to be brilliant and creative, and many people have spoken about that. You also have to be a phenomenally hard worker. And, man, he was on task all the time. It was amazingly easy to get that guy on the phone. I could get him on the phone more readily than I could get my co-author, Vicky, on the phone, and something seemed wrong about that. He was always ready to engage, and so a lot of it happened fast.
over the phone and the idea was, no, these regrets of inaction that you say are important in people's later lives, they're just sort of wistful regrets and the intense regret
Hot emotions that come from action. Those are different kinds of things. So we designed some studies where we asked people to think about the biggest regret of action or inaction. You designed the studies together. Then we ran them together, yes. So think of your biggest regrets of action or inaction from the recent past or the distant past. How many of these emotions do they lead you to feel? We had a set of five hot emotions, a set of five wistful emotions.
We thought that those long-term regrets of inaction, some of them are wistful. I wish I had learned to play the piano. But some of them are really intense. You met the right person at the wrong time, and so you let it pass. And now you're looking back like, oh, my life would have been much better if I had acted on that. We also asked people to rate whether a set of five events
I think we called them emotions of despair. I'm depressed when I think about this. I feel empty when I think about this. And lo and behold, Danny was right. A lot of long-term regrets of inaction are kind of wistful, but we were right too. They also produce these big, powerful feelings of, oh, my life is not good.
Did Danny, as a result, acknowledge that you were partially right as much as he acknowledged that he was partially right? Yeah, and that's in the paper. So truly a happy ending. I believe so, yeah. Certainly more happy than that second phone call that I got to him.
You then went on to collaborate with Danny several times, correct? Yes. I'm just curious, now that Danny is gone, and since your initial contact was this adversarial collaboration about Regret, is there anything that you now regret not having worked on with Danny?
I wouldn't say that there's a particular topic in mind, but to have someone in your life like that and not take full advantage of it, I wish I had just reached out to him more to have more contact with him. One of the nice things about this event here is this...
family feeling of all the people he reached, some of whom I know well, some of whom I didn't really know at all until now. They feel like family. So, you know, often with families, you wish you had spent more time with them. Not my family, but, you know. The adversarial collaboration with Tom Gilovich was one of the earliest ones that Danny Kahneman undertook.
Let's take a look now at one of the most recent with Matt Killingsworth from the University of Pennsylvania. He studied engineering as an undergraduate and got his Ph.D. in psychology in 2012. Danny Kahneman got his psychology Ph.D. in 1961. So how did Matt Killingsworth end up in an adversarial collaboration with this giant in the field?
Danny and Angus Deaton, both Nobel Prize winners, had this conclusion that there was sort of this plateau in people's happiness once they reached $75,000 in income. And then I had published a paper that basically showed something completely different. And we tried to figure out who's right and what's the truth. You say it with this sort of sang-froid that, you know, we published something completely different. But as you noted, you were publishing a paper in opposition to the findings of not one but two Nobel laureates. Yeah.
Were you as calm and cool about it in the moment when you decided to take up this route as you seem to be now? I mean, partly Danny is to blame, although he didn't learn that before he passed away. I had written this as like a sub point to a sub point in another paper. And actually reading Michael Lewis's book about Danny and Amos, they were talking about how they took on existing ideas. And that was an important part of their intellectual journey.
And so I sort of rolled the dice and thought, well, I'll try it. But yeah, there's definitely some part in the back of my mind of like, how is this going to go? I do have to ask at this particular conference, which is held in celebration of Danny Kahneman's work in life.
Do you feel a little bit like, you know, the wolf in the hen house or that someone's going to come up and shiv you in the back at the buffet? I hope not. No, I mean, we really had a wonderful collaboration. And I think part of what characterized that is we both just wanted to figure out the truth. I don't think anyone was attached to any particular version of reality. My sense from him is that he was a little bit irritated. Not a lot.
A lot. It was a lot. I mean, he wanted to have a good conversation, but he wasn't... For the record, that was Shane Frederick saying a lot. Barb Mellors, let's bring you in here. How did you get attached to this adversarial collaboration? Well, I was talking to Danny on the phone not too long after Matt's paper came out, and
And he said, oh, by the way, do you happen to know a guy by the name of Matt Killingsworth? And I said, well, as a matter of fact, I do. And if you'd like me to be the arbiter, I'd be happy to do so.
In adversarial collaborations, the arbiter is the research assistant, the tiebreaker, and occasionally the therapist. So, Matt, how would you characterize the response to that original Kahneman-Deaton finding about the $75,000 happiness cutoff, whether inside academia or beyond?
I mean, it was very influential, and I mean, it's probably one of the most visible kind of and exciting findings. I think a lot of us feel like money is sort of overemphasized in our daily lives, and it kind of gives a justification for maybe caring a little bit less about it and focusing more on all the other stuff that's also really important for happiness.
particularly because money is quantitative, you can kind of think of them as like points. I'm like, well, I want to get more, but you can easily imagine a sort of trap where you're just continuously trying to get more and more and more of those points, ignoring all this other stuff. And wouldn't it be nice if we could kind of step out of that cycle? At least that's part of the reason that I think that that was so attractive.
So Matt, you've rehearsed very nicely the original finding. Now bring you into the story here. What are you doing at the moment? Sure. So partly also due to Danny, my research program is really centered on large-scale experience sampling. And to get at what that matters in the original study, those data were collected how? In the original study, they basically asked, did you experience a lot of the following emotion yesterday? Yes or no? And then it had a series of emotions like,
sadness, happiness, stress, etc. People would either say, I did or I didn't. And the technology or mechanism of harvesting those data was what? I wasn't involved in that data collection. I believe those were verbal phone calls and then sort of interviewing people over the phone. Does anyone here know anything different or does that sound right as far as we know? That's right. That was said with the voice of a referee, I have to say.
Okay, Matt, continue, please. So in my study, I'm essentially measuring people's experience right in the moment. They're carrying around on their phone an app, and I'm beeping them at random times. I'm asking them, how do you feel right now? And they're responding on a scale that ranges from very bad to very good.
And the scale is continuous. So there are a couple of things that distinguish that from the earlier paper. One is that it's right at the time that people are feeling it as opposed to the day as a whole retrospectively. And the other is the gradations of feelings are on a continuous scale as opposed to something that's binary or dichotomous. And the scale is what to what? It's zero to 10, zero to 100? Very bad to very good. And it's just continuous. So you can rank it anywhere you want. There are hundreds of unique values. Got it.
and describe the differences between the pools of research subjects in the original case and in your case. In the original case, it was a survey by Gallup. It was either representative or at least an attempt to be representative, whatever random digit dialing was. That's a relative strength of their paper and of that study in general. My sample, in contrast, was really a convenient sample. It's turned out to be an amazing sample of people that have really beautiful results, but it was essentially whoever was willing to sign up to try to understand their own happiness.
What was the recruiting mechanism? Largely thanks to folks like you. People end up hearing about it in the media, on the radio, reading about it. They think, that's interesting. I'd like to learn about that for myself. I'm willing to contribute to science. And it turns out that when I look at, you know, what's the distribution of incomes, for example, in my data, it's almost a perfect match for the U.S. Census.
I can replicate all kinds of things that we've seen in the literature for decades. So it turns out to be really good, but that's kind of a lucky coincidence. Okay, so you gather, assemble, analyze your data, talk about from the analysis point to writing up the findings.
What I find when I look at this relationship, plotting people's happiness in the moment versus their income, it just keeps going up. This sort of critical point where they had found in the earlier paper this flatlining, I really see no difference at all. And when I write it up in the ultimate paper, I compare, well, what's the slope below the point where they said it stops increasing and the point above that? The slopes differ by less than 1%. I really see no evidence for a difference at all.
Were you initially looking for that plateau in the data? I was really just trying to understand what's the relationship between these things that are obviously important. We've never had, you know, in the moment continuous data. And so I want to see what does this look like? And it turns out, well, it doesn't look like what we thought it looked like. When you strike at a king, you must kill him. That's from Ralph Waldo Emerson. Matt Killingsworth did strike at the king of his realm, Danny Kahneman. But what happened next?
It's coming up after the break. I'm Stephen Dubner, and this is Freakonomics Radio.
Freeconomics Radio is sponsored by NetSuite. Quick math. The less your business spends on operations, multiple systems, and delivering your product or service, the more money you keep. Reduce costs and headaches. Graduate to NetSuite by Oracle. NetSuite is the number one cloud financial system bringing accounting, financial management, inventory, and HR into one platform. Reduce IT costs because NetSuite lives in the cloud with no hardware required. Access from anywhere.
Cut the costs of maintaining multiple systems because you've got one unified business management suite. Improve efficiency by bringing all your major business processes into one platform.
Over 37,000 companies have already made the move. So do the math. See how you will profit with NetSuite. By popular demand, NetSuite has extended its one-of-a-kind flexible financing program. For a few more weeks, head to netsuite.com slash freak, netsuite.com slash freak.
T-Mobile has home internet on America's largest 5G network for 50 bucks per month. It's how I stream the game. Oh, yeah, that one's out there.
It's how I knock out the shopping list. Hey, hon, the delivery's here. Oh, cool. That was easy. It's how I level up. Yes, double points. Get T-Mobile 5G home internet for only 50 bucks per month with auto pay and any voice line. And with price lock guarantee, your last month of service is on us if we ever raise your internet rate. Plus, there are no exploding bills or annual contracts.
T-Mobile, it's how you internet. Check availability today at T-Mobile.com slash home internet. Price lock exclusions like taxes and fees apply. See guaranteed details at T-Mobile.com. Price after $10 monthly bill credit while you maintain a nationally available postpaid voice line. Qualifying credit required. Regulatory fees included for qualified accounts. Plus $5 per month without auto pay. Debit or bank account required.
Take your business further with the smart and flexible American Express Business Gold Card. It's packed with benefits to help unlock more value from your business purchases. That's the powerful backing of American Express. Learn more at americanexpress.com slash business gold card. We've been hearing from Matt Killingsworth about his research on the relationship between income and happiness. Research that disputed elements of earlier research on the same topic by Daniel Kahneman.
So you write up the paper and the paper, not quite directly, but quasi-directly addresses the fact that your finding is contra to a significant earlier finding by significant scholars. What happens next?
There's a fair amount of attention about it and so forth. Maybe a month or two afterwards, I get a note from Barb that she's been chatting with Danny. He's talking about Barb Mellers, a longtime friend of Danny Kahneman, as well as a University of Pennsylvania colleague of Matt Killingsworth.
I think both of our attitudes was we just want to find out the truth. I have no like personal attachment to what I found particularly. And I don't, other than his initial perhaps irritation, I don't really think Danny did. It was really just what do we think is going on? Did you envision that perhaps you would then collaborate with Danny on a joint study?
That certainly seemed to be the case once we got into it. But, you know, these kinds of data took me many years to collect that data for them. They were a client on an external organization with a lot of resources. And so we ended up doing it by going back and looking at my data, which existed, which made it tractable. I can sort of cut to the chase a little bit of how we did that, if you want. I want to go back to Barb for just a second here. Barb, if you could give Danny's perspective or participation up to the point here where we're about to cut to the data.
The starting point for Danny was the assumption that both data sets were valid. So how could they be so different? There must be a resolution in the data somewhere.
And so the task was going to be Matt became the research assistant and started doing all the reanalyses on his data. How would you characterize the tenor of this adversarial collaboration? Really civil and nice and, you know, just the way it should go. So, Matt, you said you could cut to the chase. We're at the chase now. Cut to it. Essentially, the resolution was that
If we look at the range within my continuous happiness data, what if we look at the low end, which is the part where their measure would have been sensitive, can we find a similar pattern? And lo and behold, when you zero in on that, rather than looking at the average trend, which keeps going up,
you get not exactly, but very, very close to what they found. Certainly much closer to what they found than to what the average trend looks like. We both found that pretty convincing evidence that, you know, what they found is true. There's nothing wrong with the analysis at all, but it was really a question of, you know, how generally applicable is that? So from low to medium incomes, the unhappiest part of the distribution rises a lot,
And then from medium to high incomes, the unhappiest part of the distribution barely changes. But the rest of the distribution is rising steadily. And in fact, at the high end of the happiness distribution, you get an inversion of that. So rather than a rise and then a plateau, you have sort of a shallow slope and then an acceleration. So in my lay mind, as I'm trying to process all this, what I'm envisioning is that there is a cohort of people who
who are kind of cranky. And they may experience a little bit less crankiness as income rises, but there is a range in which their temperament maybe overwhelms their income. If anything, there's one of the steepest slopes for the unhappiest people in the range of low income. So getting out of poverty, if you're really miserable, is at least a
correlationally. But that's almost a different story because escaping poverty versus going from middle to upper. If you aren't poor and you're really miserable, at least if you sort of extrapolate from there, it seems to not matter past that. And probably as we speculate in the paper at that point, if you have a decent amount of money and you're really unhappy, whatever's making you unhappy probably isn't due to the lack of resources. It's something else going on in your life.
Maybe it's challenges in a family relationship, or maybe you're depressed or whatever it is. It's something going on that, you know, perhaps money isn't going to make a difference.
Richard Thaler, let me ask you a question. You've just heard Matt's presentation of his, you know, side of the original research and then the adversarial collaboration, and then the conclusion, all of which I found, honestly, to be really fascinating. You know the literature well. You, too, have a Nobel Prize, although I have heard that was a clerical error. How would you, looking at this from above, Barbara was...
there as an arbiter, Matt was one side, Danny was another side but is not present. If you could give an ultimate proclamation on not just where this finding arrived, but what the adversarial collaboration produced here, what would you say? Look, I think this is the way it should work. This is a good story. And the world would be a better place, academia would be a much better place,
if there were more of these kinds of collaborations. Would it affect what we sometimes call the replication crisis? Barbara's nodding her head no. You say oof. Yeah, let's move on to Shane. LAUGHTER
The discomfort you were hearing there about the replication crisis, that discomfort is related to a two-part series we recently produced on academic fraud, episodes 572 and 573.
And while we're on the topic of discomfort, let me add one thing I'm thinking about as we're in the middle of this conversation about adversarial collaborations. Danny Kahneman is said to have introduced this concept into the realm of behavioral research. But if you're not a behavioralist,
You could look at this as yet another example of how academic researchers discover something that people in the real world have been doing forever. There are all sorts of arbiters and referees out there, all sorts of processes for compromise and negotiation, even war games to test one plan against another and come up with the best solution.
In the same vein, some people look at the big insights that have come out of this behavioralist research as essentially Aesop's fables with a little bit of math. Earlier this year, we made a series about the late physicist Richard Feynman. He didn't think the social sciences like psychology and even economics should be called science at all. He thought they were just too squishy to deserve that name.
And as for psychologists discovering the idea of adversarial collaboration, well, Feynman was a junior member of the Manhattan Project, which in its quest to build an atomic bomb brought together the most brilliant and argumentative group of scientists ever assembled. That was adversarial. And yes, it was in the end a successful collaboration. Anyway, back to Chicago.
As Richard Thaler suggests, let's move on to Shane Frederick. He is a behavioral scientist at Yale who collaborated with Danny Kahneman on several projects and is cited throughout Kahneman's book, Thinking Fast and Slow. Kahneman saw Frederick as a protege, and they had a close relationship, sometimes a bit too close. So I'm at 10 o'clock on a Monday, maybe like third quarter of Monday night football. I get a call. Shane, he said, are you watching the game?
It was one of the rare times I wasn't. I said, no. Fantastic. I want to talk about heuristics. Another time, it was like 2 o'clock in the morning. We're going back and forth, back and forth, back and forth. And eventually I'm getting sleepy. And so I go to bed. And I wake up in the morning. And the very next email from Danny, Shane, don't play dead on me. LAUGHTER
One idea they worked on together was the Cognitive Reflection Test, or CRT. It is meant to measure a person's ability to override their gut instinct and think more carefully about a problem. Here's a famous example. A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost?
If your first inclination is to say 10 cents, congratulations. You are like just about everyone else. What the CRT measures is, can you slow down and actually calculate the answer rather than just go with your gut? If you can, you will find that the answer is 5 cents. Remember, the bat costs $1 more than the ball.
Shane Frederick has spent much of his career designing such tests. Some of the items are novel, some I invented, many of them I didn't, I collect, some I revised. One goes back to at least 1919, there are variants of it. And they were used then in an academic setting or like in employment? No, just like in books or riddles, stuff that Maya would read. But they weren't stumpers. No, no, they weren't. They weren't stumpers. I think I must tell the audience what a stumper is, okay? That, again, is Maya Bar-Hillel.
A one-way ride costs $10, a round trip costs $20, a passenger hands the cashier $20 saying absolutely nothing. The cashier knows right away that the passenger wants a round trip rather than a one-way and change. And the question is, how?
How did the cashier know this? Does anyone in the audience who does not know the answer to this stumper previously want to take a guess to the answer?
If you're stumped, good, because that's why they're called stumpers. A stumper is unlike one of Shane Frederick's CRT questions in that your intuition doesn't produce an obvious but wrong answer. It only has to do with my love of riddles, which I share with my partner in this research, Shane Frederick. And we really came to this
out of love of riddles, but we're both professional psychologists. So we felt like we had to approach it from the point of view of psychology. Now, Danny. Wait, wait. Oh. Answer, please. Oh, no way. No, I'm not going to answer. That's not what I was paid for. All right, I will answer. He handed the ticket agent two tens. Ah, very nice. Did they pay you for that?
I'm getting paid the same as you, Maya. But there was something more at stake here than answering a riddle. Bar Hillel, along with a co-author, wrote a paper arguing that cognitive reflection tests are not a good measurement of anything beyond math skills. And this argument led to an adversarial collaboration.
When I heard that Shane and Maya and Danny were having an adversarial collaboration, I didn't know who was on which team.
Maya and Danny go back 60 years. Shane is like a son for Danny. The results of this collaboration have not yet been published, but Shane Frederick says that he and Kahneman essentially proved that the CRT is a good measure of cognitive abilities beyond just math skills. It seemed to do quite well, and it's doing well consistently. Time preferences, risk preferences, other sorts of things. And
And how would you characterize the nature of the collaboration? How adversarial was it? It wasn't so adversarial between the opposing groups. It's just like everybody's fighting with everybody else about everything. More like a family gathering, right? Does this suggest that an adversarial collaboration is not as useful as one might hope if the actual adversaries are not often in the ring with the collaborators? I mean, I don't...
I don't think it will work. Do you, Barb? Do these things work? Well, what does work mean? It depends on your definition. If you think people change their minds all the way, no, it doesn't work. If they change their minds a bit, that's good.
And it speeds up science, too. Somebody's looking over your shoulder and making sure you're not making mistakes. You're defining variables more precisely. You're designing an experiment that gets right at the core issue. It's the way to go. Well, I could listen to the five of you talk for five hours. There are sessions that need to happen. This room is turning over. I thank you so much. This was a great conversation.
Sorry, there is a clause in the contract that Thaler must always have every last word. I was just going to say something nice about you, but if you insist, I'll skip it. And that was, again, Richard Thaler. I would like to thank him for putting this event together and inviting us to join. Thanks also to the other participants.
It was a pretty wonderful event, and I'm glad we were able to share it with you. And if you need more Richard Thaler in your life, and who doesn't need more Thaler in their lives, we are going to publish a bonus episode very soon, an update of a great conversation I had with Thaler a few years ago. It's called People Aren't Dumb, The World is Hard. So keep your ears out for that.
Meanwhile, on the next regularly scheduled episode of Freakonomics Radio, a close-up look at an industry that's all about close-up looking.
I like for my glasses to have a bit of pizzazz, especially if you're wearing them every day. We're going to see half of the global population being myopic by 2015. When you are a vertically integrated player, you master every step within the value chain. The margins, even by luxury goods standards, are obscene. Eyewear is a $150 billion industry, and what you see is not quite what you get.
That's next time on the show. Until then, take care of yourself. And if you can, someone else too. Freakonomics Radio is produced by Stitcher and Renbud Radio. You can find our entire archive on any podcast app, also at Freakonomics.com, where we publish transcripts and show notes. This episode was produced by Zach Lipinski with live recording by Greg Rippin.
Special thanks to conference organizers Amy Boonstra, Mark Tomelko, and Chris Partridge, as well as the Black Oak AV team. Our staff also includes Alina Kullman, Augusta Chapman, Dalvin Abouaji, Eleanor Osborne, Elsa Hernandez, Gabriel Roth, Jasmine Klinger, Jeremy Johnston, Julie Kanfer, Lyric Bowditch, Morgan Levy, Neil Carruth, Rebecca Lee Douglas, Sarah Lilly, and Tao Jacobs. Our theme song is Mr. Fortune by...
by the Hitchhikers. Our composer is Luis Guerra. As always, thank you for listening. I think my mind was wandering. I have that effect on people, I've been told. The Freakonomics Radio Network. The hidden side of everything. Stitcher.
Ross has huge savings on looks that are 100% you. So you can find all the styles that match your vibe. From stylish skorts to jersey tees. The trendiest looks of the season will have you saying, it's a yes for me. Plus, they've got shoes to make any fit pop. Be the best dress for less with your favorite picks of the season. Head to Ross and save 20-60% off other retailers' prices. Items and styles vary by store.
Don't just ride the index, seek to outperform it with Fidelity Active ETFs. Learn more at fidelity.com slash active ETFs. Before investing in any exchange traded fund, you should consider its investment objectives, risks, charges, and expenses. Contact Fidelity for a prospectus and offering circular or if available, a summary prospectus containing this information. Read it carefully.
While active ETFs offer the potential to outperform an index, these products may more significantly trail an index as compared with passive ETFs. Fidelity Brokerage Services, LLC. Member NYSE, SIPC.
Okay, you say, I want some breakfast. Your so-called boyfriend says, we got eggs in the fridge. Obviously, when you say breakfast, you mean McDonald's. Definitely a side-eye situation. Bring home the bacon, steak patty, or others with a BOGO for $1 breakfast. Only in the app. Limited time only at participating McDonald's. Valid once a day. Must opt into rewards. Visit McDApp for details. Ba-da-ba-ba-ba.