Should we be worried about OpenAI?
59:06 Share

Today's epo de is presented by S A P business A I revolutionary technology real world results. This episode search engine is also brought you by movie. Movie is accurate streaming service dedicated to elevating great cinema from around the globe, from iconic directors to emerging on tors.

There's always something new to discover with movie. Each and every film is hand selected, so you can explore the best of cinema streaming anytime, anywhere out now for movie is the substance a con Price winning sensation, delhi ious, shocking and absolutely unmissable to me. More gives a career best performance as Elizabeth Sparkle, a pastor prime hollywood alister, that turns to a mysterious experimental drug and an attempt to recapture the glories for youth is sensational.

Supporting turns from Margaret quality and hollywood veteran denis quite as a repEllent studio, exact critically adored with reviews. Having the film is in an instant body horror classic, rolling stone must be seen to be believed. Variety, the best war film of twenty twenty four world real visit.

Try the substance stock com for show times in tickets and to stream great films at home, you can try movie free for thirty days at moby doc m lurch, that's M U B I dad camps last certain infor, a whole month of great system for free. I'm going to try the substance even though i'm scared of scary movies. If you watch IT too many more.

Welcome the search engine. No question too big, no question too small. This week that we be worried about open a.

So last fall, we report a .

story about OpenAI, the leading company in artificial intelligence LED by charismatic cofounder sam open. The company was famous not giffords runaway success, but also IT for their unusual ethos and structure. Rather than simply being a four profit company. IT was a nonprofit in charge of a four profit company. And that non profit could seemingly disable for a profit company at any point if he decided that the company was acting the way that was dangerous for society.

IT was like a tech company with a dome day switch built into IT, a recognition both of A S potential power to reshape society, as well as an understanding perhaps that the last round of technological innovation has not been completely wonderful for the world. Anyway, our last story was about how opening eyes, non profit guardians had decided that the company had, in fact, gone off course in november twenty twenty three. They post their own leader. Suddenly and dramatically, the old men is out. A C E O of OpenAI announcing leadership .

transition of chat k out found IT IT looks like things .

were over for some women. Until his loyalists got on board with the counter cup. Nearly every ranking file employee at the company signed a petition demanding his return.

Ninety percent of the company, seven hundred and seven employees signed the letter, threatened to leave unless the current ford of directors resigned in reinstated altman as head of open.

Finally microsoft opening ice bigger shareholder also stepped in in supportive sam quickly there after husein data .

samon man back A C.

E O of OpenAI .

OpenAI posting on x that samon and will now officially .

return as C E O. It's also overhauling the board that fired him with new directors, ending a dramatic five day stand off that's transfixed silicon valley and the artificial intelligence industry.

So open ice rebellion board was basic, replaced with a compliant one, sam, all man who was temporarily deemed too dangerous to, on his own company inside, consulted power there. That was a year ago. In the year since, OpenAI has not turned on an army of terminators to kill us all, but the company has transformed into a summer different seeming institution with lots of strange public errors in judgment. On the way we hoped to talk to someone at open eye for the story, they did not make anyone available for comment. So instead I call the tech journalist.

now want to see something crazy.

Of course, I want to see .

something crazy. Okay.

what? What are you doing?

I just got a new webcam and IT follows my face.

but I didn't follow you. 对。 不， that a frame. And I was trying to get what til casey newton, founder and editor of the platformer news letter, whole host of the hardwork podcast. And perhaps, as sometimes too early adopter of exciting technologies, cases in the reporter resort to last year whenever things seemed be exploding at open eye and he's continued covering all the strange happenings of the company since then. I wanted to talk to him now because i'm a gossip hound for silicon valley, but because I really wondered if A I is a technology that can really change the world, how concerned should I be about some relatively erratic behavior from the company leading the field? Casey IT was happy to fill me in on what he had been going on with sam altman n in his very valuable startup since I had last wondered about .

these things twelve months ago. Well, I think on the business side, OpenAI has had an incredible year. The new york times recently reported that its monthly revenue had hit three hundred million dollars in August, which was up seventeen hundred percent since the beginning of twenty twenty three.

And IT expects about three point seven billion dollars in annual sales this year. I went back to february ary, and back then I was predicted that open eye was going to make a mere two billion dollars this year. So just this year, the amount of money they expected to make doubled. They further believe that their revenue will be eleven point six billion dollars next year. So those are growth rates that we typically see only for kind of once in a generation companies that really manage to hit on something in new and novel in technology.

And what about like how are they actually running the place? Because I will tell you, my perception as a person who follow us less closely than you is like, I feel like I see as many stories about opening eye tripping over its clown shoes as I do stories about how the new GPT is slightly Better than the one that proceeded IT like, can you give me like the time mind of last? You're like, which stories stuck out to you and how you thought about them?

So I think at a high level, and somewhat of my surprise, sam alt men changed very little about the way that he LED open a eye in the last year. Like if the concern that came up last year was that sam was not being very collaborative, that he was not empowering other leaders, that he was Operating this as a sort of very strong CEO who was not delegating a lot of power. I haven't seen a lot of change in the past year.

I have seen him continued to pursue his own piece priorities like fund raising to build giant microchips, abricotina, for example, which has a huge priority for him. At the same time, there have been stories that i've come out along the way that reminded you why people were nervous about the company last year. One that comes to mind is that IT was revealed the spring that open eye had been forcing employees when they left to sign non disclosure agreements, which is somewhat unusual, but then very unusually, they told those employees, if you do not sign this N, D, A, we can claw back the equity that we have given you in the company.

So how unusual is that? Like, how unusual is that in tech for a text to say, like like if a person quits facebook and then you'd say facebook was a bad company, how unusual to be for face. But like, we are taking back your stock.

they would be impossible to. They don't. So this is extraordinary unusual. You know sometimes would like a sea sweet executive or someone very high up in the company if they maybe, let's say, they're fired, but the company doesn't want them to run around bad mouthing them to their competitors.

They might make that person sign N D A in exchange for a lot of money. But this thing was just hitting the rank and file employees that open the eye. And that was really, really unusual. And afterwards, and all men host on on x, saying that he would not do this, and that IT was one of the few times he had been genuinely embarrassed. Ed running open an eye, he did not know this was happening, and he should have is what he said.

And just like, like, I feel like journalists have this bias, which is like we believe in transparency. We believe in disclosure. Sometimes I think non journalists care less than we do because we kind of have a routing interest and transparently and disclosure.

But it's also been really confusing, not as a reporter but just as a human being. I don't know, there's a lot of things I worry about. Most them are selfish and personnel, like what happens of over.

I is maybe in the top five hundred or a couple hundred, but there is a part remind that worries about IT. And when I worry about IT and I try to like my prediction, ledger activates, I always like, well, IT seems like a lot of people are quitting. A lot of the people work on the lets stop this from screwing up the world team but they always quit like, well, we just have a difference of agreement, can say more and it's really good. good. Yeah.

yeah, absolutely. And you I will say that there has been great reporting over the past year by other journalists who have gotten that what some of those concerns are and a lot of them why not been the same thing, which is we launched a product, and I think we should have done a lot more testing before we launched that product, but we didn't. And so now we have accelerated this kind of AI arms race that we are in. And that will likely and badly, because we are much closer to building super intelligence than we are to understanding how to safely build a super intelligence.

I see it's like what i've noticed as a user of A I, I actually noticed a safeco the other day I saw, sorry, was making a me making fun of a celebrity online. And as often happens, he says, ys, I like, didn't recognize celebrity and I plugged the picture into ChatGPT. A, I was like, who's this? Which is the main way I use ChatGPT is to say, what's this? And I was like, I don't identify human beings as okay, that's a rule that you're following. But what you're saying is that in these festivals, ts smart rules like that, which would stop people from using A I in a bad way, or stop from deciding to do things that are bad, those might be getting over ride. And that if all these companies are competing with each other to build the most powerful thing, the fastest one company, ignoring safeguards means all the other companies .

ignoring safeguards exactly. And we have seen this time and time and again, I mean, this is really fundamental to the DNA of OpenAI. When they released ChatGPT, other companies had developed large language models that were just as good, but sam got spooked that his rival anthropic, which had an la named claude, was going to release their product first and might steal all of their thunder.

And so they release ChatGPT to get out in front of claude. And that was essentially the starting gun that launched the entire AI race. And so I think IT is fundamental to how sam sees the world that all of this stuff is inevitable. And if it's going to happen anyway, all of the things being equal, you would rather be the percent who did IT right and got the credit and the glory and the users and the revenue.

So that is our overarching problem here. A I developers might care about safety, but in the rush to be first in the field, the company who wins could actually be the company who cares about safety at least, which is why we are talking about worrying incident from the industry leader. Open eye.

So one of the incident was this N, D, A incident for a reported by box this may, and the company did backtrack on those. Na, an OpenAI spokesperson told vox, quote, we have never cancelled any current or former employees vest equity, nor will we. If people do not sign a release or non disparaged agreement with the exit, then quote, a separate incident cases again, to, was the scarlet your handsome incident? Do you wanted to tell a story?

yeah. So for a while, OpenAI had been working on a voice mode for ChatGPT p. So instead of just typing in a box, you can tap a button on your phone and interact with the model using a voice. And a movie that has long inspired people in silicon valley is the Spike Jones film the poor. And in that film, walking phoenix, who plays the protagonist of that film, talks constantly to an AI companion who is voiced by Scarlett johanson.

Do you want to know how work? How do? Well, basically, I have intuition. I mean, the DNA of who I am is based on the millions personalities of all the programmer .

who wrote me. But what makes me me is my ability to grow from my experiences. So basically, in every moment i'm evolving.

just like you. And I just want to say, before you even ue with your story, what is so weird about this movie being a huge inflation people talking about? I is IT is a cautionary disp an film. I saw this movie. This is not a joke.

I saw this movie and IT upset me so much the time I was talking to a friend afterwards, and he said, I think you should probably talk to a and go on ancient president, which I did for several years. I'm not on the any longer. I went on them because of the movie, heard.

oh my god, so strange me that .

people's saw this movie and we're like, we should have this. But anyway, they love IT. They want to make IT. The future will look.

you can take different lessons from her. You know, I think a bad lesson to take would be human companionship is worthless at the moment that we invent A I super intelligence, because we can just talk to super intelligence all day long. And Turner backs on humanity that would be bad less.

And to learn, I did a lot of people in silicon value looked at her and they thought that's a really good natural user in interface, like if we could just wear ear by all day long and you can answer any questions you you ever had just by saying, hey her, what's going on with this? That would be great. And then in fact, you do start to see the arrival of products like sii and alexa and sort of baby steps toward this new world.

So I completely agree with you. Her is a destroy, an film. IT should not be viewed as a blue print to build the future. At the same time, I do feel like I see what silicon valley are in right?

You could see star wars and be like, oh, spaceships one per a pilot could be a good idea IT doesn't know you're trying to build like type fighters to take over over right? Or whatever.

And lights bors are a good idea.

and we should. I could really, really nice to think about IT so her comes out tag. People are like, oh, if you're really good to have an eye, you could talk to that's like one lesson from the movie lights aber would be good to. And when open ee releases their voice agent, which is sort of, you know, a real life version of part of this movie, the thing that a lot of people notice is that one of the possible voices for the voice agent done quite a bit. Extraction hanson, the voice for movie.

Hey, how's IT going? A rocky, i'm doing great. How about you? I'm awesome.

Listen, I got some huge news. Oh, do tell, i'm all years. Well, in a few minutes i'm going to be interviewing at OpenAI.

Have you heard of them? OpenAI sounds fairly familiar. Getting, of course, that's incredible, rocky, what kind of interview?

Not only who did the voice sound very much like garlic IT was also presented in a very fluid U. I. When they did this demo, IT was like, it's a man on using a assistant who has the voice of a woman who sounds a lot like very lo hanson and she's like, O, P, J, you're so bad that was was like the toof IT. And I was kind of like, what are you doing here exactly?

After the product launched, a user on tiktok even asked ChatGPT itself if IT pulled IT was a johanson in clone. Is your voice plus to be surgham? My voice isn't designed to replicate scarlet joe hanson or any specific person whole. Seriously, the voice has never sounded more similar to joe hanson to me than when I was denying the resembLance cases, said the company itself had also contributed .

to this confusion. Sam alt had primed everyone to think that way because a couple days before they do this demonstration, where they show off the voice for the first time, sam altman tweet the word her, or I should say, he posted on ex. And so of course, when this demo happens, everyone is like, oh. And so everyone was sort of prime to think, oh, wow. OpenAI has realized silicon valleys decade long dream of making the movie her a reality.

And then what happens then?

IT turned out that scholarship hanson was really mad because sam alt men, i'd gone to her last year and said, hey, would you like to be a voice for this thing? And he thought about IT, and he said, no, I don't want you. And then apparently after he had posted like and just in a couple days before the demo, he'd gone back to her agents and try to renegotiate at this whole thing and said, are you sure you don't want to be the voice for this thing and SHE said, no and they showed IT off anyway and they never said, this is scarlets ohs in but they absolutely let everyone believe IT a new controversy tonight in the world of artificial intelligence .

as one of hollywood s biggest movie stars as her voice was copied without her consent by one of the most powerful A I companies actor Scott johanson claim ed open a eyes ChatGPT minute voice for its latest personal assistant program this bizarre moment let to scarlet johanson then making the rounds on TV advocating for legislation to protect the intellect al property. Really the identity of actors like yourself.

Obviously, we're all waiting and supporting like this, like the passing of legislation to protect everybody's individual rates. And I think you know it's yeah but like still waiting for IT, right? So like until this is just maybe sort of highlights like how vulnerable everybody is to IT, I think this was the story for me of all the stories that like really stuck with me.

And maybe because the message he gave me was a kind of immunity. And like the promise, as I ve ve understood from voting an eye, has been exactly the opposite of immunity. And obviously, like of all the choices they make, whether they find a sound like voice actress and do a voice that sends a lot like sergio hand and then kind as much as the truth, I can see a person getting over enthusiastic and making that mistake.

It's the kind of mistake, uh, podcast are making in first couple years ago. S I got were really sorry, but IT seems careless also, this is a product where one of people's concerns is the copyright implications, where these are companies are who wearing up a lot of people's creative work to make their products. And IT just felt like what you expect from a company that doesn't care what you think and wants to do what he wants. And I don't know, i'm over reading, but there was a moment that kind of lag gave me a little bit .

of future logic. I am with you yeah, I think you framed IT really well because this is the company that has told us from the beginning we're working on something very powerful. We think I could solve a lot of problems if IT falls into the wrong hands, IT could also be extremely dangerous. And so that's why we're going to come up with a very unusual structure for ourselves and trying to absolutely everything we we can do in our power to proceed safely, causal, ously and responsibly. And so you look at the Scarlett o hand thing, and like none of that's where is what their behavior in that case.

So that was the scarlet o hanson incident. Casey told me about another incident, this one from this past August. Let's all that on the lazy student problem.

And this is a kind of short and funny one, but there was reporting this year that they built a tool that detects when students are using ChatGPT to do their homework, but they won't release IT.

How do they explain why they're not released him as someone who has had to have a conversation with a teenager about why they shouldn't SHE using open eye and really stumbled on the part, I was like, listen, it's the wrong thing to do and you probably won't get caught. And also, yes, probably all your friends are doing IT. And then like, there was like several allies of laws why I realize like the whole I G up into, why won't they just released the homework?

I should say the walls we journal broke the story and the statement they gave to them was the text water marking method. Were developing a tactically promising but as important risks we're weighing while we research alternatives, we believe the deliberate approach we taken is necessary given the complexities involved, and it's likely impact the brother ecosystem. Bedd open an eye, that is what they said. The journal sort of made an alternate case, which is that if you can use ChatGPT to treat on your homework, you will stop paying the company twenty dollars a month.

Is nobody to imagine what part of their revenue is coming from high scholars and college kids? And also like I don't know, maybe there's an argument that just like the same way we don't need to do long division. Nobody needs be able to think a reason in S A form. But I think people people think A N A form.

So if we're trying to decide if we trust open an eye to be not just a profitable company but also a kind of unusually ethical AI standard barrer they're willingness took accept a bunch of grubby twenty dollar bills from high schoolers who wanted skip their homework and play more fortnight. It's not the end of the world. But IT.

Is behavior unethical enough that you'd probably fire a babysitter over IT? Casey offered to tell me about an additional incident that had given ten people pause. The investments incident. This one had to do the same altman personally, specifically the way he's been quietly spending his money investing in companies like stripe, R B N B and read IT.

We did learn about Samuel man's investment empire this year, thanks to some reporting in the all strict journal. They may really dug into all of the stakes that he has in many startups and found that he controls at least two point eight billion dollars worth of holdings and he's used those holdings to create a line of debt, uh, which he has from jp work and chase, which gives him access to hundreds of millions of more dollars, which he can put into private companies.

And like, why is this interesting? Well, one that's kind of a pretty risky gamble to you have a lot of your network tied up and like dead that you raised using your venture investments as collateral like that kind of a rigid latter of right there. But that also creates questions around what companies is OpenAI doing deals with? Are those companies that sam has investments? In course, sam doesn't owe equity in OpenAI right now, and so his own wealth is tied up in these investments. And while nobody really thinks that sam is doing any of this for the money, there was just the kind of also this finishes al element to what we learned about him this year that I think raising questions for people.

I feel like one of the things where I feel a little bit distributed as I think a couple years ago, I hadn't made up my mind, but I felt very willing to entertain the possibility that sam altman was a very unusual kind of person, that he didn't seem to be motivated by accumulating wealth to the same degrees maybe other people are, that he might not be entirely moderated by accumulating power, that you might just have a vision for a technology that could be really useful or could be really dangerous, and thought he might be the best person to be a start of dad.

Not saying I was right then. I'm not saying I was wrong then. But like, do you feel like you have a changed or refined view of what motivates this person who has a lot of power?

I essentially have the same view of his motivations, and I think the generous version of IT is that he is in a long line of silicon valley entrepreneurs who thought they could use innovation to solve some of the world's biggest problems, and that that is how they want to spend their lives.

I think the less generous version of IT is that this person coming out of that tradition found himself working on this technology that could essentially be like the technology that ends all other technology, is because if the thing works out, the thing you created just creates all other innovation automatically for the rest of time. And that is the position of extraordinary power to put yourself into IT. And I do think that he is attracted to the power and the influence that will come from being one of the people that invents this incredibly powerful thing.

After a short break, casey ari mentioned that there have been a lot of senior level departures at over. I we're going to dive deeper into who left and what they seem to believe about the company they were quitting. Pus will look at a fairly worrying warning, manifest sh by an x opening employee.

The just matt.

Today's episode is presented by S A P business A I revolutionary technology real world results. Hi bai, hello.

It's so nice to meet you. I recently is go to a .

listened in north CarOlina who emailed us about life at her job. SHE was calling us from outside her office.

inside her car. My friends, though, is lurking outside because she's the one to me, to the .

podcast.

So she's looking at a least excitedly and giving me strange .

looks that so funny, that so funny. Dais, a web designer for a company that helps promote events for clients all over the world, he told me about some ways SHE used as a eye to shorten her work day.

There are some fun use cases, like extending images a little bit larger and photoshop. There is languages that are like non lain that have funky characters with those spaces like japanese. And sometimes I have to do translated websites for events that are happening, say, western happened. I have to include manual breaks for the lions, because if there aren't manual breaks, the meaning will change. And so I will sometimes use A I to help me find the best breaks that can be there, every possible spot.

So you using IT, not your translation, but to make sure that you're not like starting a new line in a way that would totally change the meaning of try to communicate.

Yeah, exactly.

And did you learn the hard way that the line break can change the meeting .

everything a little bit? I know we send our sites to a client in like a local office to someone who spoke japanese and IT didn't go the best. They had to resort back and say, like me, this doesn't work. This isn't gonna OK daily .

adverse thought he was going to have to learn rudie entree japanese. But then you realized software could actually help her out here. No need to learn a new language.

I get to spend more of my time doing the actual, like creative intensive in us parts of web design versus the meal, kind of repetitive parts.

Thank you so much for talking about this. Yeah, should we wave to?

Oi, oh, absolutely. wait.

Where did he go?

He might be hiding behind my car.

Thanks again to bail y for chatting with us and zi for so expertly hiding your by ready to elevate your business with S A P business A I you can grow revenue, increased efficiencies and manage risks seamlessly, is relevant, reliable and responsible. A I is embedded into S A P solutions to businesses to drive immediate business impact with A I embedded across their organization. IT can also help you make confident decisions based on A I grounded in business data and put A I into practice with the highest ethical security and privacy standards. S A P business A I revolutionary technology, real world results .

this show is sponsored by Better help. This month is all about gratitude. And along with the person I just shout IT out there's another person we don't get to thank enough ourselves is sometimes hard to remind ourselves that we are trying our best to make sense of everything.

And in this crazy world, that is an easy here's a reminder to send some thanks to the people in your life, including yourself. That's where Better help comes in. Therapy from Better help can assist you with everything from learning positive coping skills and how to set boundaries, empowering you to be the best version of yourself.

If you're thinking of starting therapy, give Better help a try. It's entirely online, designed to be convenient, flexible and suit your schedule. Just fill out a brief questionaire to get matched with a license therapist and switch therapies any time for no additional charge. Let the gratitude flow with Better help. If is a Better helps that com slash search engine today and get ten percent off your first month, that's Better help H E L P dot com slash search engine.

Welcome back to the show. So if you like me, were at best quarter paying attention to development at opening the past month, the thing you still may have noticed was just a very unusual amount of senior level people leaving their jobs. IT was a kind of turnover ver you'd expect to see at a halloween store in november, not typically at one of the most valuable new american technology company.

We've already mention this, but OpenAI employees were in many cases, discouraged from criticising the company. And yet there's still been some evidence about why they left and what they saw before they did. So we're going to get into that. This part is not so much an incident as IT is a series of incidents, a trend, let's call this bit sudden departures.

So the first big one at the door this year is the sky. Andrey carpathians is part of the founding team he'd left for a while to go to tesla. He comes back for exactly one year and .

then leaves.

okay. In may, ellia succored, who was one of the board members who had forced sam out last year, he announced that he is leaving the company and doesn't really say much about why he's leaving, but within a month is revealed that he's working on his own AI company called safe super intelligence and raises a billion dollars just to get them off the ground.

Oh well.

yeah. He had a guy on his research team name, yang likee. So this was somebody else he was trying to make sure that AI is built safely. He leaves to go to and propt to work on that problem there.

Gretchen ruger, who's another the researcher, leaves in may that in August, john shuman, who was one of the members of the founding team, he announced that he was going to anthropic and he had previously helped build ChatGPT. And then greg rock, man who is the president of OpenAI and one of its kind of main in public face s spokespeople, he announced that he is taking an extended leave of absence. Basically, he says he really needs a break.

Not entirely sure what happened there. And then finally, mirror, I announced that he is leaving in september. SHE had also been part of this board drama last year.

And at the same day that he left, IT was revealed that the companies 的 chief research officer， bama, grew and another research VP Better off or also leave in the company. That's just a lot of talent walking out the door. pj. And I can say if you look at the other major AI companies, so like a google, a meta and anthropic, there has been nothing comparable this year in terms of that level turnover.

So you have a huge turnover at the top of a company that, in theory, people should want to stay out because it's like leading the industry. It's incredibly valuable. It's the winning team and people are walking on .

the doorways and they don't want to play for IT. Yeah totally. But you know another really important story about memory is that but sam was ousted last year.

He had written a private memo to sam raising questions about his management and had shared his concerns with the board. Oh, interesting. And my understanding is that, that would weigh heavily on the board when they fired.

And because to have the co of the company coming to you and saying, hey, this is a real problem yeah, that's gona get your attention in a way that, you know, maybe a rank and file employee might not have been able to get their attention. So we have known for some time now that mirror has had long standing concerns with sam management style. And so when he finally left, IT felt like the end to a story that we event following for some time.

And so has he said anything publicly that is very defer able about her reason for acting?

So, you know, he said this, never an ideal time to step away from a place. One cherish es, which I felt was just an, acknowledged that this seems like a pretty bad time to stuff away. But he said that he wanted the time and space to do her own exploration. And on the day that we recorded this, the information reported that she's already talking to some other recently departed OpenAI people about potentially starting another A I company because is what people do, like most people, when they leave OpenAI, they start an AI company that looks shockingly similar to open a ye .

just without sam. And why is that? Well.

my glib answer is that the high ranking people who leave OpenAI seem to feel like the problem with open A S sam altman and that if you could build a eye without sault man, you would probably be be having a Better time. I I see. And then there's one other guy who laugh that I want to talk about, yeah, is a guide name.

Leopold asan brunner.

Okay, you heard this guy? No, i've not. So he is a quite Young. He's still in the twenty. He was a researcher at OpenAI.

He is fired, he says, for taking some concerns to the board about safety research, opening item this. But he goes away and he comes back in june. And he published as a fifty thousand words document online called situation awareness. Were you aware of situation awareness?

I was not aware of situation awareness.

okay. I'm here to make you aware of situational awareness. It's this very long document that was the talk of silicon valley for a week or so and in a live hold says essentially the rest of you out there in the world don't seem to be getting IT.

You don't understand how fast AI is developing. You don't understand that we're actually running out of benchMarks to have IT below past. And this technology really is about to change everything just within a few years.

And IT sure seems like outside our tiny little bubble here, not enough people are paying attention. And this document winds up getting circulated all throughout the bite and White house IT is been its circulated in the trump campaign. And I think liao action brunner might, in a trouble administration have talked himself into a role like leading the homeland security department or something. But yeah, he was another one of the interesting .

departures this year. That's a crazy dark ment. Like what do you make .

about I think that while you might take issue with some of his logic and some of his graphs and maybe his hand waving past certain potential limits in the development of this technology, he is getting at something to real, which is that IT does seem like even though A I is essentially topic number one in tech, IT doesn't feel like people are really reckoning with the potential consequences maybe as much as they should have.

Yes, some people may listen to this and say, well, you know, casey is sort of fall. And for all of the happier now, there remains this contention of people who believe that this whole thing is a house of cards and that once the successor to GPT four comes out, we will see that the rate of progress has slowed. And in fact, no one is going to invent super intelligence anytime soon.

And all of these things are just gonna sort of wash away. IT might just be an effective who I spend my time with. And the conversations that are happening at at dinners and drinks and cement tesco every day. But I am more less persuaded that we are very close to having technology that is smarter than very smart humans in most cases, and that if you are the person who controls the keys to that technology, that, yes, you will be extraordinary ly powerful.

Listening to casey, I started to imagine a potential world where A I continues to grow at whatever pace that grows out. But we're opening ice squanders its early lead in the industry and just becomes less important over time. I wanted to know what casey thought of this possibility. Do think there's a world where open eye becomes less important to the future of this thing and you know will end up talking more about these other companies because these other companies have absorbed so much of the talent of that place.

yes. And there's actually this really fascinating president for this in silicon valley. So we call silicon valley, silicon valley because I was where the semiconductor industry was founded and the biggest early semiconductor company was called fairchild. And much like open eye in the early days of chip manufacturing and attracted all the best talent, but one by one for various reasons, a lot of people leave their child and they go on to start their own companies, companies with named like intel.

And there are, why not being so many of these companies that they start calling them the fair children? Because they were born out of this initial company that sort of seated the ecosystem with talent, made some of the key early discoveries and then lost all that talent. My guess is you I didn't know the name fairchild before I said IT just now, but you do know the name in tal. And the question is, due anthropic and some of these other upstarts become the actual winners of this race and open a ee fifty years from now is just a fit note history.

So how much should we be worried about open eyes? I guess the answer for now seems to be someone, if you think A, I really could be powerful and if you think A I safety is an important IT doesn't really seem like the incentives in a race to dominate the AI market are really that a line opening. I might end up leading the field.

IT might end up being a fair child. But it's hard to imagine why any AI company that would succeed while also moving forward with a bunch of cautious, at least not without some regulation. After a quick break, we're going to switch tracks a little bit.

We talked a lot about why this technology, maybe concerning a lot of people, agree so much so that on some quarters of social media, you can get shamed just for using AI products. But I am one of the people who both worries about A I and uses A I. And in last year, as the technology has got a much more powerful, I find i'm using IT in stranger ways. When we come back, i'm going to to casey a little bit about how he thinks about the ethical concerns here and also about the very bizarre he has become talking intimately with a machine. That that to so much.

Welcome back to this show. So I wanted, as casey about this, A I question i've been personally conflicted on and remain somewhat person ally convicted on. It's the first time in my life i've seen a new digital technology some people despise so much they don't want to use A T at all.

I see people shame each other online for using A I at all. And that feels like a very online response to something. But IT doesn't feel like a strategy. But I also like understand where the impulse me comes from.

Like how do you square IT for yourself where people's jobs are important, people having jobs is important? All that money just sort of getting swept into a big pile. OpenAI doesn't feel like totally socially at the same time, like I used ChatGPT. It's not replacing anybody's job in my usage of IT. But I don't think as IT became more useful, there be a point where I would say it's a moral for me to use that.

I'm gonna stop. yeah. I mean, we have always is used software tools since their advent to try to automate away ery, and that has actually been seen as a good thing, right? It's nice that you have a spread sheet to do your financial planning and aren't trying to do at all on a legal pad president that brought a benefit to your life, make you Better at your job and also helps you do IT faster.

And I use the A I tools I use as doing that. They take something that used to take me a lot time and effort and now make IT simpler. Er for just one example, like I have A A human editor who reads my column before I stand IT out, but I also will most of the time just run IT through claude actually, which is on proper x model.

I just see if I can find any spelling or grammatical air. And everyone once in a while IT really saves my bacon and all cost me is twenty dollars a month. So I don't think there is any shame in using these tools as a kind of backstop to provide you for making a mistake or you know, from doing some research because that is the way that we've always used software and technology.

So I understand the anxiety about this. I understand people who, for their own principal reasons, to decide what I don't want to use this in my work. Maybe i'm a created percent.

It's very important to me that all the work that I do is one hundred percent human and has no A I and IT. These are like very reasonable positions to strike. But I think that he tells someone you shouldn't use this particular kind of software because IT is evil.

I don't understand that. 你们 can I tell you about another web and using A I this year？ yeah. And I was actually thinking about you because during one of our conversations, we were reflecting on the fact that there were only a couple of things that people could do to improve their mental health. And one was therapy and the other was meditation.

And you will think how frustrating IT is to know what the answer is and to not what do IT right? It's like, yes, if you started a meditation practice like that would obviously be very helpful. But then you have to, like, sit quietly with your thoughts for twenty minutes to daily.

Obviously, that seems horrible. yes. So recently i've been experiencing these feelings have burn out related to my news letter, where I love doing IT, but IT also feels harder than IT has.

And i've been doing IT at least three times a week, sometimes as many as five for seven years. And so I think this is just sort of a natural thing. And so I felt like I need to maybe break last in the case of this emergency, and try something that I never previously wanted to do, which was meditate.

Oh, oh.

so i'm only a few days into this. I don't want to tell you that i've sold anything here. I didn't enjoy my first few experiences, but one of the things that I did, both in the run up to and the aftermath of these meditation experiences, was to just chat with claude, be as claud lets you create something called a project, where you can upload a few documents and you can chat with those documents.

And then you can just also have chicken with IT from day to day and tell you what you're noticing or observing if you have questions. And to me, this was a perfect use case for this technology, because I truly know nothing about meditation. I know people have talked to me about, know, done a couple of times before, but i've never read book about IT.

I've never talked with any of my friends at length about IT, so i'm just as fresh as you can be. And the level of knowledge that is inside cloud, which was opposed to stolen from the internet without being anyone from their labor, yes, is actually quite. And IT was able to help give me a good start.

And then afterwards I could come back and say, well, you know, here's what I noticed. I struggle with this thing. I say, oh, you might want to try that.

Or, you know, I sort of wish was a little bit more like this. And I say, oh, well, you might want to try the other kind. Tell me more about that. Okay, yeah, sure. Here's everything.

And I was talking earlier about like, what will you be like when you have an AI coworker is like, well, I have a meditation coach that I paid twenty books a months for. Some people are laughing. Some people are in case you can mediate for free.

You don't need a coach. I get that. I am somebody who likes to like paying for access to exercise. And I feel like a habit. And first of all, I am going to go meditate after this because I want to restart myself and I didn't get to do IT this morning.

I don't know if i'm still gonna doing this in like two or three weeks, but if I am, I think the AI is actually to be part of that story because it's giving me a place right and go after these experiences to reflect again. When I hear people saying, casey, you realized the journals exist, you could like write this down. But yes, I I get what you say.

This is a journal that talks back to you. This is a journal that is an expert about the thing that I am journal about that is holding my hand through a process. None of this existed two years ago, right? totally.

The chAllenge of talking about any of this stuff is when the rate of change in your data day is high. Sometimes that feels quite obvious. Other times IT becomes a weird blind spot where you don't even realize that the conditions around you have changed, right? This is what leaps is getting in situation.

Awareness is like you need to stop and collaborate and listen as valise. What said what you're doing on this podcast? P, J, which is like, it's Better a year.

What happened? This is the right question, right? You know, we are talking so much earlier about these AI critics that are like, it's all hype.

It's constantly wrong. Screwy silicon valley road. I take my totally, get all the enemies and resent that the power's that. But something that those folks do to their detriment is they tune out everything that is happening in A I, because they think i've already made up my mind about this stuff.

I already know that I hate everyone involved, I hate the output, and I hope IT chokes and dies right now. This is how these people feel. And I, again, I get, I understand all these motion. What i'm saying to you though is you actually have to look around, you have to engage, you have to keep trying out these chat pots every two or three months, if only to get a sense of what they can do now that they can do two to three months ago, because otherwise you are going to miss what is happening here. And IT is wild.

IT is wild. It's to me is really interesting that IT is in a strange way, a tool you are using to know yourself. And I don't mean to overstay IT like IT is also just a journal that is talking you and giving you the pointless. But like I find that interesting.

I also feel like for whatever reason, I think because there's such a culture of like we don't want to be enthusiastic about technology any more, particularly the technology which you don't want to end up looking like a person who was gliding lily, celebrating the arrival of our doom. And so there's kind of a weird lack of just like ten years ago, I think had this come out, there would be a tech Price that would say here's ten ways you can use this. Here's how i'm using IT kind of nobody wants to be seen doing that's not using that.

I had a thing happened a couple of years ago. I think sam altman n he was reaching someone whose suggestion was like, ask your agent from all of our interactions. What is one thing that you can tell me about myself that I may not know about myself? And I asked this question and I got an answer.

And IT wasn't like a fortune cookie. Horoscope like vega. IT would apply to anybody and maybe be views for anyway. Like IT was a real thing that I had known. I was like the preponderance of your questions me are about trying to put structure and precision around processes in your life that do not have them. You are constantly asking how long things should take and how much time to allocate IT is clearly something you're strugling with.

Wow.

which is the kind of thing like a good friend would tell me you and IT is not an experience. I've had the software and I don't know, like I find myself in amount or i'm trying to hold everything in my head at the same time to say these are technologies. We should be skeptical, love. And to your point, keeping engine to and also um in the time before this possibly changes the world ways I might not enjoy pretty useful. Absolutely absolutely.

I mean, it's interesting because like .

I think you're right. I think we have always use software to automate jury. And one way you could think of that is like, IT does all mate human labor and the people who have I jobs and i've had dreddy jobs aren't like i'm so glad that i've been free to produce anything else there like upset that the vanguard is being taken from.

Why do you think? A I is the place where these anxiety finally come to ahead? Because in previous errors of software, whatever skepticism people had about IT, this skepticism m actually .

feels new to me, has a great question. And I think there's a lot that goes into IT, I think, that were living at a time where there's kind of low water mark and trust in our technology companies. I think the social media era really destroyed most of the good will that silicon valley had in the world because people see these technologies like.

Facebook and instagram is tiktok is mainly just things that like steal our time and reshape the way we relate to each other in ways that are obviously worse. And the whole time, the people building these technologies insist that actually that they're saving in the world and that there's nothing wrong with them yeah and another generation comes along and says, oh, hi, we are actually here to invent god. There's going to be a lot of there's gna be a lot of scupham about that.

And you know IT is the AI companies themselves who told us this thing will create massive job loss. IT will create massive social disruption. We may have to come up with a new way of organizing society when we are done with our work.

That is something that every CEO of every AI company believes. B, J. Is that we will have to reorganize society because essentially capitalism, what makes sense anymore.

So most people will agree that, like, they don't like change. You know, change is bad. And when they say they don't like change usually means what I have a new manager at work. The change that these people are talking about is that capitalism to exist anymore. And it's unclear, like.

is somebody because everybody that I mean this be a little of broadly many people in our generation are like, I would love for capitals. And is this anymore by which they don't mean robots do the work now and robots are your boss and robots take all the money and you're hoping for maybe universe no one meant forget to go away like, yeah yeah exactly nobody .

wanted capitalism to go away and be replaced with something where a silicon valley seem to be in control of everyone's future.

right? And so we continue to pay attention this because, well, who knows how true these problems will come? The idea that this is socially disruptive seemed like a safe fat yet.

You know, maybe something else to say that important is that the way all of this is unfolding is anti democratic, right? No one really asked for this. And the average person does not get a vote, right?

If you're just like an average person, you don't want A I to replace your job. There's nothing, nothing you can do about IT. And so I think that actually breaks a ton of resentment against these companies.

And while the government is starting to pay attention, least here in the united states, they're being very, very gentle about everything. And so if you wanted to change the course of A I, it's not actually clear how you would go about that. And so I think that's another really big reason why people often resented.

And it's funny. There's always apart in my mind when you see these stories of these departure is to say, okay, that's like the internal drama of a company that I do not have an internal view on. And I might matter, might not.

I would have to know more than I know to know, but do. What your point is, if part of the problem is that these technologies can research a society, we have democratic society. But the way the researching society is not democratic than the fact that even within these companies, they are becoming more like monarchy. Does he make .

something for intentional? Yeah yeah.

Cassie newton, he raised the newsletter platformer, go check that out. You also have listen to him every week on the podcast hardwar. We're going to keep using you to monitor this.

Yeah let me just say i'm going to keep paying attention to casey.

Thank you.

You're welcome.

Sir chene is a presentation of odessa and jizo productions is cleared by me to vote in truthy plimmer I, and is produce by garages m and no john back checking this week by mary maths theme original composition in mixing by armed bizarre an our executive producers are generates burman and lee Dennis thanks to the team at jigsaw alchemy, which pillow a and to the team at od jd cri, rob, andy, crag cox, r.

Doni, kate rose, mac casey more occurred just a Francis curt corti and hills v our agent is oran rose bomb A U, T, A. If you would like to help support the making of the shop, if you would like to vote for our existence, you can turn up for a premium subscription at search engine dot show. You'll get add three episodes of the show as well as the ocasio bonus.

You can follow and listen to search engine with pj. Vote now for free on the odessa APP or or if you get your podcasting. Thanks for listening to see next week.