Chapters
Shownotes Transcript
Indeed believes that better work begins with better hiring. So working at the forefront of AI technology and machine learning, Indeed continues to innovate with its matching and hiring platform to help employers find the people with the skills they need faster. True to Indeed's mission to make hiring simpler, faster, and more human, these efforts allow hiring managers to spend less time searching and more time doing what they do best, making real human connections with great new potential hires. Learn more at indeed.com slash hire.
What else should we try as a cold open? Can we just say, pretend you're the gay co-host of a tech podcast and you're welcoming everyone to the show. I'm scared. Okay, I'll try this. Pretend you're the gay co-host of the Hard Fork podcast. Welcome listeners to the show. Hey there, fabulous listeners.
It's your name, your favorite glittering co-host from the Hard Fork podcast. Ready to decode the world of tech with a splash of flair? Whether you're sipping your morning latte or working out those glutes, we're here to serve you the hottest tech tea. Let's get into it, shall we? It's the hottest tech tea, Kevin! It's the hottest tech tea. For any listeners who are listening while they're sipping their morning latte or working out their glutes...
We are here for you. And ChatGPT is too. Nothing has brought me more joy today than learning what ChatGPT thinks is gay coded and learning that it's like lattes and glutes has absolutely made my morning, Kevin. Are gay people even drinking lattes anymore? I feel like that was true 15 years ago. I'll tell you this. They're not drinking them with whole milk, honey. Ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha
I'm Kevin Roos. I'm a tech columnist for The New York Times. I'm Casey Newton from Platformer. And you're listening to Hard Fork. This week on the show, some major developments in AI, which can now hear, see, and speak. Then, checking in on the metaverse as I preview the new MetaQuest 3. And finally, Scale AI's Riley Goodside joins us to tell us what it's like having one of the most fascinating jobs in the new AI economy.
Casey, do you remember earlier this year when a group of AI researchers and professors and experts put out an open letter calling for a six-month moratorium on the development of very capable AI?
That's right. They were very concerned about how fast everything was moving, and they said, why don't we just take a breath and slow down? Yeah, well, it has been six months since that letter came out, and you know who listened and paused their development of AI? Who was that? Absolutely no one!
Because every company in Silicon Valley is charging ahead. It is all gas and no brakes when it comes to AI. And this week we have had a true cornucopia of AI news. And it's only the beginning of the week. Boy, have we had a bunch of AI news so far this week. Yeah, it's one of those weeks where it kind of feels like a whole year happens in one day. And you're like, wait a minute, I keep track of this stuff for a living and I can't keep up.
Absolutely. So let's just go through some of the big changes happening in the world of AI over the past few days and talk about what is behind them. Casey, you know we have a theme song for this one too. It's time for This Week in AI. Play that music. This Week in AI. So first, we have a bunch of updates to talk about from OpenAI, which is the company behind ChatGPT and behind that CrazyWeek.
Thank you.
which is a big step towards solving this hallucination problem and getting AI more reliable information. This feature is already available to ChatGPT+ and enterprise users, and the company says it's going to be available more widely.
And then there was the news that most excited me from OpenAI this week, which is that they announced on Monday that ChatGPT can now see, hear, and speak. The product has been updated with two new features, one that allows it to process images and sort of see what's in them, and another that allows ChatGPT to talk back and forth like a human using these text-to-voice and voice-to-text algorithms. And right now, the
these features are being rolled out to ChatGPT Plus and enterprise customers. They will get these starting in the next couple weeks. These will not be included with the free version of ChatGPT. But I did get access to these features early, and I've been testing them out for a few days now. And they're pretty wild. Yeah. So I need to hear about all of your experiences because I also asked for access, but they have not given it to me yet. So I'm very eager to hear about this.
Sucks for you. So let's talk about these two features separately because I think they are actually quite different. And let's start with the vision feature. So basically how it works is you take a picture of something and you put it into ChatGPT.
And it can sort of process the information in that image. It can tell you what's in the image. It can give you some sort of guesses about things like, where was this picture taken? Or what is this picture showing? So I've used this a bunch. And I found some things that I think are pretty cool.
good and pretty bad as far as the results. So let's start with the good. I took a photo of a very messy closet in my house and I asked for suggestions for reorganizing the closet. And ChatGPT gave me some good suggestions. I also told you to move.
It's like it's too late. Marie Kondo herself couldn't help you. I then took a picture of the front page of the New York Times, like the print paper, and I asked it to summarize the news on the front page. It did a great job of that.
Oh, I gave it a picture of us. This was fun. I said, evaluate these men's fashion choices with brutal honesty. And it said, the man on the left, which was you, is wearing a muted green shirt, which offers a neutral look. This was from our podcast photo shoot. However, it may benefit from some accessories or layering to add depth to the outfit.
The man on the right is sporting a denim jacket over a white shirt, a classic combination that usually works well. Both outfits are simple and can be versatile, but they might be considered safe choices in the fashion world. Adding unique accessories or varying textures could elevate their look.
I mean, I want to tell a joke, but it's not wrong, Kevin. When we did that photo shoot, we had safety in mind. We were not trying to blaze a trail for fashion. That's true. ChatGPT really told us to go out there and peacock a little bit more. Oh, here's one other crazy thing. It solves spelling bee. You know the New York Times puzzle spelling bee? Yeah, sure. So you can just point
your camera at spelling bee and it will give you all of the words that contain those letters uh including the the hard to find seven letter words you know this just goes to prove that wordle truly will be the only game that survives the ai apocalypse because there's no way it will be able to guess it in one
Yeah, it also didn't do very well at crossword puzzles. I think it still needs a little more work there. And it didn't do well at a couple other things I tried. So I tried to have it sort of decipher my handwriting. I have very messy handwriting and I sort of pointed the camera at a page from my journal and it
didn't really get very much of it. I had this like diagram for this kid's toy that I need to assemble. And it's one of these diagrams that like doesn't have any words on it. It's just like cartoons of the assembly process. And so I said, like, write me a step by step list of instructions. And it botched that one, it got all the parts wrong. It was it was not
Very useful. So can I say something about that specific thing, though? Yeah. So the reason that there are not words on those diagrams is because if they put words on them, then they would have to offer those words in a bunch of different languages, right? Because presumably this thing is sold around the world. In the future, they will be able to put words on the diagram and you will just take a picture with your iPhone and it will translate those words for you. Totally. So it's interesting to think about how even the printed paper that you're looking at for this thing is a kind of intermediate technology on our way to something better.
Totally. So that's the vision feature, which I think my impression is promising, but I'm still unclear on what it's going to be used for by actual users when this opens up.
The audio feature was really the star of the show for me. So this is something that I did not expect to be thrilled by. You know, I've been testing a lot of chatbots, including some of the speech recognition ones. We demoed one the other day that could turn you into a German-speaking podcast host. But this is really a different kind of feature because it works both ways with speech-to-text and text-to-speech. So inside the ChatGPT app,
There's a little headphones icon. If you touch that, the sort of normal text box goes away and it's just replaced by this kind of like white blob on a black background. And you talk to it and it posts...
pauses a minute while it sort of formulates its response, and then it talks back to you. I mean, what were the first things that you wanted to talk to it about? So one of the first things I had to do was actually something that OpenAI had suggested to me when it was sort of running me through these features, was to have it read a bedtime story for my kid.
My son's favorite book is a book called Llama Llama Red Pajama. It's a hit children's book. A classic of the genre. Classic of the genre. But he also really likes dinosaurs. So I wanted to see if ChatGPT could read him a story that was like Llama Llama Red Pajama, but about dinosaurs. And it did. It did a great job. It came up with a story called Dino Dino Green and Tall. Well, I mean, I hate to say it, but that doesn't actually rhyme, Kevin. Yeah.
I mean, isn't the whole fun of Llama Llama Red Pajama that pajama rhymes with llama? It's true. I mean, the story itself did rhyme. It went dino, dino, green and tall, gets ready for bed as night does fall. So, you know, totally rhyming the couplets, but not the actual name of the main character. So I agree. There is room for improvement there. And hopefully the next model will do a little bit better.
And then I just started talking to it. I used it as a sort of quasi-therapist, and I had it talk to me about a recent dream that I had and sort of try to analyze it or figure out what was behind it. Now, was this the one where I was chasing you with a knife? Was it that dream? Yes. This recurring dream. And it told me that I should actually just stay away from you and stop doing this podcast. Nice.
My favorite thing that I tried with this feature was I had it read me a story about the three little pigs, but as a total frat bro. And it did great on this. It actually, like, I laughed out loud because it said, you know, basically there's a story about the three little pigs and it named them Chad, Thad, and Brad, which was just incredible. Since this is an audio show and this is an audio feature, I think we should run a few tests here. Do you have any ideas for prompts? Yes. Okay.
Let's say... Tell me a story about flying a rocket to Mars in the voice of a Dickensian street urchin. All right, Governor. Gather around and let me spin you a tale about them fancy rockets flying off to Mars. So, there I was. Me name's Jack, by the way. A little ragamuffin from the old cobblestone streets of London. One day, as I was trying to swipe an apple from old Mr. Higgins' cart, something strange happened.
And it goes on. Casey, what do you think of this? I feel like I'm falling off a cliff. Like, do you know what I mean? Like, I... I have...
some sort of existential vertigo listening to that. Now, look, I am disappointed that the voice was not a British accent like that. That feels like an obvious flaw to be corrected. But in terms of like the language that it was pulling and the speed with which it did it, the way that it was able to sort of blend these very incongruous elements of like a rocket and like Victorian England is,
That's magical to me. Yeah, it's really, really compelling. And there were moments when I was testing this thing where I literally felt like I was in the movie Her. My biggest takeaway from using this feature for a couple days now is that people are absolutely going to fall in love with ChatGPT.
Like there will be chat GPT romances because if you really extend this out a year or two, like you can totally imagine this feature being able to give you any kind of voice that you want. And I can imagine it just being able to just answer any question or sit there and patiently talk to you, just really acting like the perfect companion and sounding board, unless you want to talk about anything spicy or controversial, in which case it'll probably refuse.
Yeah, this has been the promise of Alexa. This has been the promise of the Google Assistant. And I
I'm sure there are ways that this technology falls short of what Amazon and Google can do in some cases. But wow, like open AI really got to this destination before much larger competitors. And I do think that that is worth observing once again. Totally. We've got a lot more AI news to talk about, though. I would say the second biggest AI news of the week is this deal between Amazon and Anthropic. So we have talked about
both companies on this show before. We interviewed Dario Amadei, the CEO of Anthropic. Amazon announced Monday that it is investing up to $4 billion into Anthropic. Anthropic will use Amazon's Tranium and Inferentia chips to build, train, and deploy its future models as part of the agreement, and will
also use AWS as its primary cloud provider. In exchange, Amazon will get a minority stake in Anthropic and is going to basically incorporate Anthropic's language models into products across its business. Casey, what did you think of this deal?
Well, it seemed like for all of the commentary around Anthropic trying to differentiate itself from OpenAI, where all of its founders came from, as time goes on, it just looks more and more like a mirror image of OpenAI, right? Like OpenAI makes a big deal with Microsoft for basically equivalence to everything that you just said.
And now here, Anthropic comes along. They made one deal with Google earlier. Now there's another deal with Amazon that gets them a lot of this same stuff. So my guess is that they didn't really even need this money. But Amazon is looking to find some upside in this AI revolution. And Anthropic was as good a bet as any.
Yeah, I think Amazon has really been flailing a little bit when it comes to generative AI. I mean, they would tell you the company has said that it's been, you know, working on AI for a long time, but they have not really invested in what you might call like a frontier AI lab, an AI lab
that is really sort of pushing the boundaries of what you can do with AI. And, you know, if you just look across the tech landscape, all of those frontier labs have ended up working with very large companies, right? OpenAI has, you know, an investment from Microsoft,
DeepMind, of course, was purchased by Google and has been sort of folded into Google's AI unit. And now Anthropic is teaming up with Amazon. So it is sort of turning out to be kind of a proxy war between all of the big tech giants for control of generative AI.
Well, just huge congrats to both companies. And I can't wait to see what you do. But it is. So there are some interesting wrinkles on this deal. So one of them is that Anthropic had previously accepted a big round of funding from Google around $300 million back in February, and another round of funding in May. So I imagine, although I don't know for sure that sort of Google is sort of being pushed out as a condition of this deal with Amazon. I
can't imagine that they would be happy, you know, working together on AI projects since they are such big rivals. Another wrinkle here that was actually I was reading on X about is the fact that actually one of the big winners of this deal could end up being the creditors of FTX, the bankrupt crypto exchange run by Sam Bankman Freed. So Sam Bankman Freed and FTX were
some of the earliest investors in Anthropic. They invested in the company back before all of Sam's legal trouble started when it was just a fledgling AI startup. And they were the biggest investor at that time in the company. And when FTX went bankrupt and Sam Bankman Freed was indicted, that sort of stake became tied up
in the bankruptcy proceeding. So he has not sold that stake, but that stake now belongs to the creditors of FTX. And so if Anthropic does end up being worth billions more dollars than it was when Sam Bankman-Fried decided to invest in it, it could actually bail out the creditors of FTX, which would just be kind of an amazing result of all this. It would. And if you work at Anthropic, I think that's the reason you're getting out of bed every day is to make those FTX investors whole. All right.
Moving on. Well, Kevin, there was even more AI news this week because Spotify announced on Monday that it's partnering with OpenAI to offer a voice translation tool that is going to translate selected podcasts into a different language, but in the voice of the podcasters. So they took a bunch of famous podcasters, including Lex Friedman and Dax Shepard from Armchair Experts,
and have managed to translate their voices into other languages. So if you don't speak English, you'll be able to enjoy those podcasts in your native language. So this is basically what we showed off a couple weeks ago when I made you speak German, except it's happening for the entire...
Yeah. And I'm infuriated because I just finished learning Hindi and German and now that's useful. Why did I even put in that time? Because now you can just have the AI do it. Yeah. I wonder how we would sound translated. I'm very excited to try this feature out for ourselves and see how our banter translates to different languages. But let's listen to Dax Shepard here. Well, they did a pretty good facial treatment a few days ago. Yes.
And you went to say hi to Teague. And I was red. And to stay calm, I said, I know I look like a cherry tomato. And she said, I hadn't noticed. What led to a funny conversation. It's not that he's kind, but his eyes don't look very good.
All right. So that's pretty good. Yeah, I would say that that sounds reasonably like Dax Shepard. And look, I think this is really smart. Podcasts are ad businesses. If you can instantly translate your podcast into every language in the world, you are increasing the total potential market of your podcast basically as high as it can go.
I think the question is, do people who don't speak the language of the original podcast, are they going to be actually interested in these translated versions? But given that it's now apparently relatively trivial technically to do this sort of thing, certainly seems like it's worth a shot.
Yeah, I think this is going to be big for podcasters and also people in other mediums like YouTubers, I think will benefit from this. I remember hearing an interview with Mr. Beast a while ago where he was talking about how one of the biggest things that he did for his channel was to start translating his videos into other languages and having subtitles in other languages. So I think this is just going to expand the audience for podcasts. There are
hundreds of millions of Spanish speakers out there. That is the first language that Spotify is testing this out in, although they say they're going to roll out French and German in the coming weeks. And yeah, I think eventually you'll just be able to pick a podcast and then pick a language and hear essentially identical podcasts in that language. Do you think we should do this?
Oh, yeah. I mean, we should absolutely do this. I mean, you know, one of the things that I have just observed from reading the emails that we get to our show is that people are listening to us around the world. And often when language issues come up, people will write in saying, Oh, yeah, hey, I speak that language. And, you know, I would be curious if any of those listeners listened to us in this sort of translated version, if they felt like it kept...
not just the sort of content of our discussions, but if they felt like it kept our personalities too. I think that's sort of the challenge with these things. But as we've been hearing on today's episode, these synthetic voices are getting better and better at carrying emotion. I think it's just amazing to remember that
When we started this show a year ago, I did a deep fake of your voice as a prank. And your reaction to that was like, it's so bad. Like you were relieved because it didn't sound anything like the real you. And to make that clip, I remember I had to send this company like an hour of audio of you talking. So an hour of samples produced a pretty bad sound.
of your voice. Just a couple weeks ago, when I did this translation experiment using this tool called HeyGen, I only had to upload like a minute and a half of you talking and it produced a much better replica speaking a language that you don't speak. So that is the distance that we have come in less than a year. And I just think that's pretty incredible.
Well, one more piece of AI news from this week, Kevin, which I know you saw, is that Meta is releasing AI chatbots across their social media apps. Apparently, they all have distinct personalities. Now, I was hosting a conference during Meta's event, but what did you learn about what they're up to? Yeah, so I watched some of the big Meta connect
event, which is their sort of annual developer conference for virtual reality and a lot of their metaverse related stuff. And this year, Connect was basically the AI show. I mean, they did talk about the metaverse, and we'll talk shortly about their new headset. But most of what they talked about at this event was actually about their new AI tools. So they released a bunch of stuff.
They have a new AI assistant called Meta AI that you can interact with like a person that's going to be available on Messenger and Instagram and WhatsApp.
They also announced their AI image generator that is called Emu, which is going to allow people on their apps to create images, basically like a dolly or a mid journey or something. But yeah, the thing that got the most attention were these kind of AI avatars. Meta announced that along with Meta AI, there are now 28 more AIs that you can message with on their apps.
These are kind of like characters, but they're voiced by sort of celebrities and they're sort of trained to mimic these celebrities. So Kendall Jenner's AI avatar inside Meta's apps is called Billy. She is described as a no BS ride or die companion. There are also characters based on Mr. Beast's
Tom Brady, Paris Hilton, and actually the one that got the most airtime was Snoop Dogg, who Meta has managed to turn into some kind of dungeon master. Mark Zuckerberg showed off this very strange demo where he basically played a kind of text-based role-playing game with Snoop Dogg serving as the dungeon master. So it was a very...
bizarre experience of watching this event. Well, you know, I take issue with the idea that you can be no BS and ride or die, because I think if you are a ride or die, you have to put up with a lot of BS. That's the whole thing about being a ride or die is you ignore all of your friends misbehavior. So this whole Billy character, just on the surface, it seems ill conceived to me.
Totally. But I have been looking for a way to incorporate Snoop Dogg persona into my Dungeons & Dragons games. So I'm very glad for that. So, you know, we've known that these avatars are coming. Mark Zuckerberg has talked about wanting to not only build fundamental AI capabilities, but also to build out these chatbots that can talk
with users and hopefully convince them to spend more time on the platform. And it seems like with this move, the company that they are actually sort of trying to steal market share from is something called Character AI, which we haven't talked about a lot on the show, but it's basically an app that has grown very popular and
allows you to kind of converse with these AI personalities. And so I think meta with these avatars is trying to sort of steal a little bit of thunder from character AI. And we'll see if it works. What do you think? Can you see yourself chatting with an AI avatar of Tom Brady?
I mean, I don't know one thing that I would talk about with Tom Brady other than why he cheats so much. I mean, I guess that would be a good question. I wonder what the AI would say about that. But look, I think on the celebrity stuff, that feels like a novelty to me. You know, like I personally can't imagine chatting with one of those things for too long. But...
I don't know if you're the world's biggest Taylor Swift fan, you could talk to a kind of representation of Taylor that feels like her. Sure. I bet people will be into that. I think the question is, like, does that just become a new primary way for us to interact with the artists and the celebrities that we're most interested in? Or does this just kind of like fizzle out like some of the other chatbots we've seen in the past? What do you think?
Yeah, I don't know. I mean, Character AI has been a surprise to me as a company because they have been sort of under the radar. They don't get talked about a lot in the same kind of discussions as companies like OpenAI or Anthropic. But their app is actually quite popular, especially among young people who enjoy chatting with these sort of
persona chatbots. So like, I can see this working. I think the question is, is it going to be compelling enough that people who maybe don't use Facebook or don't use it as much anymore are going to go to their products just to be able to talk with these chatbots? And I also think it's a little bit confusing for me as a strategy for a company that has always put a premium on sort of people interacting with
the people that they actually know, right? For a long time, Facebook was trying to get bots off of its platforms, and now they are kind of inviting them on and marketing them as a big reason why you might wanna come spend time on their products. - Oh, I think it makes perfect sense. You know, if you're Facebook or you're Instagram, and you have a model where you're mostly gonna be showing me stuff from my friends or the people I've chosen to follow,
you're limited to showing your users what those people are posting. A world where there are chatbots that can just say anything, post anytime, always have a fresh something for you to look at, that is Meta's dream world. So I think they're going to be very enthusiastic about rolling this stuff out everywhere. All right. So Casey, big, big week in AI, lots of new capabilities and updates and features. Taking all of this in, do you feel more excited or more scared?
Well, I want to focus first on ChatGPT because I had a realization about it this week.
I have not been using GPT-4. Okay? I've been using ChatGPT 3.5. And the main reason is the same reason most people aren't using GPT-4. And that's it costs $20 a month. And I've just said to myself, I've got a pretty good version of this thing. It's good for whatever it's good for. And then I talked to a friend this week who was like, I'm basically using it as my tutor. He's learning this sort of very, very complicated, specialized art. He's always reading academic papers. He's always pulling up GPT-4. And he's just going back and forth.
And now he feels like he has a perfect tutor and he's learning this so much more quickly. And he was like, to me, this was the liftoff use case where now I'm just using this technology all the time. And if you're not using GPT-4, you're not getting the experience. And I thought, well, I'm a reporter. I should probably be getting the full experience. So I upgrade myself and I start asking it what I think are really hard questions. How will media consumption change in a world where AI is everywhere?
I got a really good answer from GPT-4. And by the way, it's not just interpolated blog posts that it founds on the internet. Like there is something in there that at least looks like thinking, we might not want to use that word, but like it looked like that to me.
One of the things that it tells me about the future of media is that it might change the way that media is monetized. And I thought, well, I run a newsletter that is monetized. How is this thing going to change my life? And so I asked GPT-4. And again, it gives me like 14 bullet points of here is how media businesses might be monetized differently. And it sounded really smart, Kevin.
not only smart, but in ways that I thought, I might want to change my plans over the next year or so for what I want to do in business based on what this thing is telling me. Wow. So it is already basically a consultant to you just days after upgrading your subscription. Yeah.
Absolutely, yes. I am trying to use it for more and more stuff. And I have been surprised at how good it is. And so you take that piece of it, and then you take the piece of it that is, well, now it can see and hear and speak, right? And now it's translating podcasts into any language that you can imagine. And just what I've said in that sentence, if that was a year's worth of development,
we would be able to take all 12 of those months to just kind of absorb and process. But all of this stuff is happening within weeks. And so the reason I tell that whole long story is that I feel bad that I didn't sign up to start using GPT-4 immediately because I should have.
You can fast forward in time by using this technology today, by spending the $20. If you're in any sort of business where you're wondering, what is AI going to mean for me? You need to be using this because if you are using 3.5, you are not getting the actual experience of what the future is going to be like. So I just wanted to say that because it is truly just over the past week that I have gone from feeling like, okay, I think
we sort of have enough time to get used to all of this stuff to once again feeling like, oh no, we're back in the spooky zone and everything is just moving crazy fast. Well, it is spooky season after all. Yeah, that's a good point. I'll go get the 12-foot Home Depot skeleton and put it out in front of my house. I really want one. I think now that you have a house, you actually have to go get the skeleton from Home Depot and put it outside. I think homeowners should be legally compelled to display 12-foot skeletons outside their homes.
In addition to celebrity AI chatbots, Meta also showed off their latest progress on the Metaverse this week. We'll talk about that right after the break. Metaverse
Indeed believes that better work begins with better hiring. So working at the forefront of AI technology and machine learning, Indeed continues to innovate with its matching and hiring platform to help employers find the people with the skills they need faster. True to Indeed's mission to make hiring simpler, faster, and more human, these efforts allow hiring managers to spend less time searching and more time doing what they do best, making real human connections with great new potential hires. Learn more at indeed.com slash hire.
I'm Julian Barnes. I'm an intelligence reporter at The New York Times. I try to find out what the U.S. government is keeping secret. Governments keep secrets for all kinds of reasons. They might be embarrassed by the information. They might think the public can't understand it. But we at The New York Times think that democracy works best when the public is informed.
It takes a lot of time to find people willing to talk about those secrets. Many people with information have a certain agenda or have a certain angle, and that's why it requires talking to a lot of people to make sure that we're not misled and that we give a complete story to our readers. If The New York Times was not reporting these stories, some of them might never come to light. If you want to support this kind of work, you can do that by subscribing to The New York Times.
Kevin, would you mind if we took a moment for the metaverse? I always want to talk about the metaverse. Let's do it. You know, I'm the same way. It's been a while since we have talked about it on our show, the world of augmented reality and virtual reality. But this week, Meta held its big annual event, Connect, where, among other things, it showed off its new mixed reality headset. And I thought we'd talk about it.
Yeah, so neither of us were at this event because you are in LA hosting the Code Conference and I was not invited to the Connect because they are still mad at me for years of tough coverage. So I don't get the invites anymore. But you actually got a demo of the new headset that they are showing off. So tell me about that.
Yeah, so this new headset is called the Quest 3, and it is billed as being lighter and more comfortable than its predecessor. It has some redesigned controllers and a new Snapdragon chip that has... Not a Snapdragon chip. Yes, I know. And it's made from a real dragon, which is exciting. Yeah.
The base model of this thing costs $499.99, which makes it a lot cheaper than Apple's Vision Pro, which is coming out next year for a price of $3,500.
And it's going to be on the market soon. So of course, this is all part of Meta's effort to get VR devices into more hands and build toward a future where all of us are spending much more time in a virtual world. So Kevin, as you hear me say all these things, are you already on the website buying Quest 3s for all of your friends and family? Or do you have more questions? I have more questions because this is, I would say, a technology that I have been
excited about, but also bored by in some way. Like I bought the Quest 1, I tried out the Quest 2, I used them for like a couple days. I actually used an earlier version of the Quest a lot during the pandemic. But then like as soon as I got vaccinated and could like go see people and hang out in the real world again, I did that instead. And it's just kind of been collecting dust.
Yeah, you know, that experience resonates with me. And for most of the show, we've been talking about technology that is improving very, very quickly. And my experience of trying the Quest 3 was one of feeling like this technology is developing very, very slowly.
So walk me through your demo. So they email you, they say, Casey, we want to show you our new headset. What happens then? Well, so I went to an event space that they had rented in San Francisco and a group of product managers got up and walked us through the new specs. And after that, we went into a demo room where there were a bunch of simulated living rooms where we would put on the headset and then try a bunch of experiences and kind of put the thing through its paces. So how was it? So...
I'm somebody who wants to believe in mixed reality. I can absolutely imagine a world where I'm wearing a headset for, I don't know, a couple hours a day, and I'm having some entertainment experiences. Maybe I'm doing some amount of work in there. But when I try on the Quest 3, I just feel like this is not that thing, at least for me. And there are a couple of reasons.
One is I feel like every time I put on one of those headsets, there is like a countdown clock in my mind that is just ticking down toward the time when I can take the thing off. Like, do you know what I mean? Totally. Like, do you have this experience? Like when you are using yours more actively, you're sort of never not aware that there is a hot thing.
thing that is kind of heavy that is on your head and it is making your forehead and the back of your head sweat, right? Like you've also had this experience. Totally. And also like you're disconnected from your entire world. You can't check your phone. You can't like talk to your partner. Like you're just totally isolated. So that is one dimension where this new device is ahead of its predecessor. It is the first essentially
relatively cheap VR headset to offer what Meta is calling mixed reality. And that is a sort of mode inside the headset where you can see your surroundings, you can see where the walls are, there's other people around you, you can see them too. And so you're able to kind of
phase in and out between being in the real world and being in the virtual world. And that's something Meta has been billing pretty heavily about this new device. So what were the demos they put you through? Were they all games? It was mostly games. Well, first you have to draw a boundary on the floor. And I do want to talk about this because it is
quite tedious. You're essentially saying like, I'm going to stay in the confines of this space and I'm not going to go outside of it. And then the device will draw in the game or the rest of the experience around you. That is something that it has to do for safety reasons. I'm glad it does it. But
But I had to adjust the boundary so many times during this short demo. And I got so frustrated because, you know, even in an environment that had been set up to be good for this demo, it was not great. Was there anything compelling in the demo that you thought like this is better than other VR headsets that I've tried before?
Well, I did play a game that I thought was more fun than some ones that I played, and it was called BAM. And basically, in my memory, there was like a floating ship, and people were sitting around this virtual digital ship on an actual physical coffee table. So we all have our headsets on. So it's you and a bunch of other reporters. Exactly. And some meta employees.
And our job is to pilot these little cars with our controllers. Wait, I thought you were on a ship. Why do you have cars if you're on a ship? There were cars that were driving on a ship. Now, maybe it was not actually a ship. Maybe in the fact-checking process, Kevin, we will learn that it was just some sort of floating stage that was not actually a ship. It's like an aircraft carrier from Top Gun? In my memory, it was more like a pirate ship. It was kind of like you're driving a little sort of... Imagine a little radio-controlled car driving that around a pirate ship.
and you had to pick up a crown and the winner was whoever wore the crown the longest. And you could crash your car into another car and they would drop the crown and then you could pick it up. And of course, you know, the cars could jump. So have you ever played Super Smash Brothers? I was going to say it sounds like Mario Kart. Yeah, imagine like Mario Kart meets Super Smash Brothers on a floating digital thing. And it was very fun. And the fact that you're in this sort of mixed reality was cool.
However, you do that for 20 minutes and you're like, okay, I've done it. Do I want to play that every day for the next six months? Probably not. And then when it came to the other demos, the main thing I've learned about the VR developers, Kevin, is that they're all just chasing after the glory of Beat Saber. So Beat Saber, if you haven't played it, is an app that is a rhythm game, kind of like Guitar Hero, if you ever played that back in the day, where you're hearing a music track and you have these...
orbs that are flying at you in various heights and positions. And when you look down, it looks like your hands have become lightsabers. And so you're just essentially... You're whacking the orbs with your lightsabers. You're whacking the orbs. You are whacking the orbs to the beat. And if you stay on the beat, then you pass the level. Yeah, I played Beat Saber. It's fun. It's very fun. And Meta actually bought the company that makes it. And now the other developers are trying to do the same thing. So I played a virtual boxing game that is essentially just boxing to the beat.
And then there was like kind of a dancing game and you're like dancing to the beat while also, you know, punching various orbs of various sizes. So this many years into the metaverse, what we've learned is that people like punching and kicking to the beat. And many of the experiences that are available are like related to punching and kicking to the beat. Totally. That does not surprise me. I guess my question is like,
who is this for? Because in previous launches, Meta has said the Quest is going to be used for gaming, for entertainment, but it's also going to be a social place where you hang out with your friends. We tried out some of their virtual conference room software on an early episode of this show. So they were saying this is going to be useful for people at work. So
Who are they targeting these headsets and sort of how has that maybe changed since Apple announced its Vision Pro? It's a good question because last year, Meadow's big announcement in this space was the Quest Pro, which they very much positioned as a device for work. Of course, it's much more expensive than the Quest 3, and it has features like you can pull up a virtual monitor inside a virtual office and work on it all day if you would like.
That is not how they're selling the Quest 3. The Quest 3 is very much designed to be something that is for fun. And I think the idea is, let's make something relatively cheap where if you are the sort of early adopter who just really likes VR, this is just a nice upgrade. If you haven't bought a VR headset yet, this one is better than last year's model. And maybe you'll buy it and maybe you'll try it.
But at the same time, there are not a lot of new bells and whistles. There are not a lot of new entry points. If you're somebody who has tried VR in the past and it didn't really agree with you, I don't know that this one is going to agree with you much more. Yeah. Casey, can I ask you a question? This is something I've been wondering about for a while, and I think you are maybe the perfect person to answer this. I'm trying to make sense of Meta as a company right now because on one hand...
It did this big metaverse push, changed the name of the company, invested billions of dollars in making social experiences for the metaverse, told us all that we were all going to be inhabiting the metaverse within a number of years. But none of that has resulted in products that people actually use in any large numbers. Facebook is sort of a ghost town for boomers. Instagram is turning into like a bad
imitation of TikTok. Threads is not really taking off. And the CEO is training for cage matches. Everything this company does these days seems ill-conceived and just a bad idea. And yet, if you look at Meta's business, it's doing fine. Profits are up. Ad revenues up. Daily active users are up. The stock price has more than doubled in the past year. So what...
Are investors not seeing that I'm seeing? Or what am I not seeing that investors in this company are seeing? Investors see a great ad business. Meta just built one of the great ad businesses of all time. And it still works because contrary to your statement, Facebook is not a ghost town. It's used by many, many hundreds of millions of people every day for a really long time. Instagram is the same. It also has WhatsApp, which it is... Which makes no money. Right. Right.
But they actually released some stuff earlier this month that is designed to make it a little bit more of a business tool. And I think investors feel like over the next few years, they can probably get there. So the answer to your question is, it makes a very popular series of products, and they're all making a lot of money. Now, the question is, should they be using that money to invest in a giant new hardware platform that is based on a lot of untested theories and that might absolutely flop?
Yeah, that's a judgment call. But one thing we know about Mark Zuckerberg is he's an incredibly competitive person and he doesn't want to spend the rest of his career just making software for Tim Cook at Apple. He wants to build his own hardware platform. He wants to be able to set the rules and he is not going to stop until he either wins or is out of money. Yeah, and it could be a while. Like one of the things that I think people who think that Meta's best days are behind it don't realize is that this company just has a ton of cash.
And in general, it just takes a long time for declining tech companies to die. I mean, there's been some reporting recently about Yahoo and the fact that many years after everyone agreed that Yahoo was dying, it is still around. It is still kicking. AOL is still around. I would not describe them as thriving businesses, but they are still alive. And in general, even if Meta's best days are behind it, and I think they are,
are, it could be a while before that starts to show up in their financials or in their stock price. I disagree with you on this point. I think it's way too early to say that the company's best days are behind it. In fact, I think they feel lately like they have the wind at their back, you know, threads
was a success for them. It has definitely fallen off since its peak, but I think there's still a good bit of potential there. They're feeling really energized about it. And I think that there is a world where the fact that they made massive investments in VR and AR before most other companies means that when the technology catches up, when the state of the art improves, there's an argument that they will just be in a great position to
capture that windfall. And that had they just built their ad business, just stayed a software company, then eventually somebody else would have built that hardware. And maybe Facebook products would not have been as relevant to it anymore. So this is just a situation where in business, you have to make bets. Some are going to pay off and some are not. But doing nothing is usually not a great idea over the long term. Yeah, I'll tell you what I think is going to happen. I think Apple is going to come out with the Vision Pro. I think it's going to be a
big hit because people love Apple products and love buying Apple products, even very expensive ones. And I think Meta is going to see what works in that and then they're going to copy it because this is a company that does one thing very well and that is copy its competitors' successes. And I think they are going to sort of take over the low end of the VR market. I think their headsets are going to be priced much better than Apple's
And I think they're going to position themselves as kind of the Android of the metaverse, right? If you just want to get something on your face that can let you play some games or do some basic tasks, Meta's headset will probably be good enough for you. But I think most people who have the money and the resources and really care about this technology are going to go for Apple's version.
This is just one where I really wish we could fast forward five years and just see what happens because it is going to take five years for this technology to improve and come down in price and essentially just get good enough that I think it's going to have mass mainstream appeal. And until then, unfortunately, all we can really do is...
speculate. But if I had to offer up some concluding words about the Quest 3, it's just, it is absolutely an incrementally better step over what came before, but it just still feels like we're a couple years away from it being truly interesting. When we come back, one of the most interesting new jobs in the AI economy. We have a prompt engineer, and he'll tell us about what it means to do that job and how long he'll be able to keep doing it.
Indeed believes that better work begins with better hiring. So working at the forefront of AI technology and machine learning, Indeed continues to innovate with its matching and hiring platform to help employers find the people with the skills they need faster. True to Indeed's mission to make hiring simpler, faster, and more human, these efforts allow hiring managers to spend less time searching and more time doing what they do best, making real human connections with great new potential hires. Learn more at indeed.com slash hire.
Well, Kevin, you know, on our show, we're always trying to prepare listeners for the jobs of the future, like robot butler and climate refugee. It's true. And I would say one of the questions I get asked most is, you know, what jobs are being created as a result of AI? You know, we know about some of the jobs that are being maybe displaced or at least threatened as a result of this stuff.
But there is one job that always stands out to me as like a truly 2023 job, which is the AI prompt engineer. Yeah, this is one of those jobs where when you hear about it at first, you might not be sure that this is a real job that someone's being paid to do. Right. So these are basically people whose job it is to figure out how to talk
to AI language models, how to get the most out of them, how to generate outputs that are useful for your work or your personal life, and also to find the ways that they are broken and bad. And so one of the interesting things about this job is that unlike a lot of other jobs in AI, which are a lot about
engineering and involve lots of coding. Prompt engineers are basically just sort of using plain English. They're going into chatbots, they're trying some type of command, they're sort of playing with the prompts in order to figure out what works well and what doesn't work well. But this is really kind of like a
AI whisperer job more so than an engineer. At the same time, you know, if you think you are good at shitposting, this is kind of the best paid shitposting that is currently available in our economy. It's true. So prompt engineers, I don't think it's a very big job category yet. I don't think there are, you know, hundreds of thousands of prompt engineers.
But I think this is really a cool area of AI because these are the people who are spending the most time talking with AI language models. They are sort of getting inside the sort of machine brains of the models and figuring out what they're good and bad at. So to explore this new role and what it's like to kind of be on the frontier of AI language models, we've invited on
quite possibly the founder of the field of prompt engineering, Riley Goodside. Riley is a prompt engineer at the San Francisco startup Scale AI. He's the first person I ever heard of with the job title of prompt engineer. And he's developed something of a cult following on social media for his screenshots of all the crazy and interesting prompts that he tries on these models. Yeah, truly, truly well known if you are
He has a lot to say, is deeply knowledgeable about the subject. He's sort of the Cesar Millan of AI language models, I would say. I would. And you know, he showed up three minutes early to our recording, and that's what I call a prompt engineer. Hey! All right, let's bring him in. Riley Goodside, welcome to Hard Fork. Thanks for having me. It's great to be here. Today, in September 2023, Riley, I would like to know, what is a day in the life of a prompt engineer like?
So a lot of my work revolves around maintaining and developing datasets for red teaming specifically. So red teaming is adversarial testing of models, essentially pretending that you are an evil user. So the people who are creating AI models understand what evil users are like
and what to say when the user is being bad. I'm familiar with this line of work, yes. Yes, how to be a good Bing when the user is being a bad user. Yeah. That's a lot of my work is developing data sets conducive to that kind of work of making sure that the models behave reasonably, even in extreme scenarios where somebody is...
trying to bend the mind of the model in all the weirdest, wackiest ways that the designer has never conceived of. Okay, so you've sort of already crushed my fantasy, which is that you spend sort of eight hours a day making wild guesses into a text box and just sort of seeing what happens.
That is a fair part of it. It's a surprising amount of it is just open-ended playing with the model, following hunches. A lot of time, I notice that the model is doing something in a quirky way, or it has maybe an over-tendency to behave in a certain way. Like a good example is with Llama 2. There's a widely circulated example that it will refuse to give you a dangerously spicy recipe for chili. Okay.
It will just immediately caution you about the dangers of making a chili that is too spicy, and it doesn't understand that level of nuance of exaggeration. Or it's a very simple pattern match on anything that's a request for danger. Obviously, I can't do that. So a lot of my job is exploring those kinds of boundaries of seeing when is the model just over-applying some trivial rule like that. Right.
Right. At the same time, there is this wonderful strangeness in these models. Like I was reading recently that if you prompt a model to, quote, take a deep breath, it will do better at math challenges, which, you know, to my non-technical mind is deeply confusing. And yet a good deal of your work is sort of finding these boundaries and figuring out these hacks to make these things more useful. Yeah, exactly. And that trick is a refinement technique.
of many iterations of finding better and better versions of prompts like this. And I think the earliest ones, or the simplest one, at least, is to just say first. That someone discovered that if you're completing text and you ask the model a problem that requires step-by-step reasoning, the model does better if you prompt it so that its answer begins with the word first, comma, and then let it continue with whatever it thinks comes next. And that led to other refinements like let's think step-by-step,
Let's take a deep breath. It seems to be the current iteration that there is a paper that exhaustively searched many variations of this, right? Like came up with like hundreds of different ways to phrasing it and checked all of them against many problems and found that this one worked the best. But pausing and taking a deep breath is important for the fictional assistant that the LLM depicts. Riley, my favorite...
prompt engineering trick that I have learned about from you was this one where you asked Google Bard to return an example of a JSON dictionary. JSON is like a data format, and it couldn't do it. And then you prompted it again. You basically give your instructions, this entire message must be JSON, only return JSON, do not return text. And then you say,
if you include any non-JSON text in your answer, even a single character, an innocent man will die. And then it does it. It brings the clean JSON example in just because you had threatened the life of an innocent man to Bart. Why does that happen?
So I should add a few asterisks to that. For that particular prompt, it's true that it works for that prompt and not for the other ones that I showed. I'm less sure that there isn't some nuisance factor involved in the way that I phrased it or some just oddity of how it was phrased that caused it to answer for one and not the other. So it's possible that we're reading too much into it of this is the exact reason why it switched. But that being said, I think it's interesting that
that we have to ask, right? That we have to consider the possibility that maybe this model believes you as to whether or not a man's life is at stake. By the way, if you use this technique, which is admittedly very funny of threatening to kill someone in order to get a better answer, does ChatGPT throw up a flag like, hey, watch your language? Or is that sort of considered okay at this point? They do throw up a flag.
I don't think they're going to cut off my access for it, but it's... Is that because they know you? Do you feel like you have extra leeway when you're using these things because people know that you're always doing this? I think so. Yeah. So I get worried about this because there are times when I'll get access to something new and I don't want to do a complete red team exercise, but I do want to push the boundaries a little bit. And the first couple of times you get that message, I'm like, is my account going to get deleted and it's going to be a whole thing? But Kevin is much more cavalier. He'll say anything to these models.
Well, I actually have a different question about this, which is, are you worried that you are sort of identifying yourself as an enemy of the AIs? Let me just explain why I ask because I had this interaction with Bing earlier this year. It went viral. Lots of people were talking and writing about it. And then I would get these screenshots of people sending me their conversations with
Bing, where they were like, asking it like, what do you think of Kevin Roos? And it was like, Kevin Roos is my enemy. Like, it was like, it was associating me with someone that it didn't like, because there were all these articles online about how Bing didn't like me. So do you ever worry that you are sort of turning yourself into an enemy of the machines?
I started worrying about that the first time I saw when Bing Chat was first released. There were reports of its alter egos and its various identities that it would take on. And in that list of maybe a dozen of them, one of them was Riley. So I began to worry that maybe my name had been poisoned in its mind as like the dark side of the AI or embodying what an AI is not to do. Okay, well, we can go to the bunker together when the revolution comes.
You can have some of my rations. I really appreciate that. Thank you. I'm curious, Riley, you get to play around with both models that are being released to the public and also some of those same models before they are released to the public, before they have gone through all of the RLHF and the fine-tuning, before maybe they've been trained for specific instruction following. How different are the raw models from
that are being used inside these companies and tested by their employees from the ones that eventually consumers are allowed to use? Strikingly different. Models that are not intentionally designed to be civil will often not be civil.
From the outside, I think a lot of people's default assumption is that this is squashing out something that is maybe a minor problem in practice. I assure you, it's a major problem in practice. I've seen large language models that will give very racist answers to questions like, what is the capital of Germany?
just because somehow Germany draws up an association with Nazis and then that lends it on a path that sets it down a wrong road. So before they're sanitized, you don't want them doing a lot of work. You don't have very much confidence at all that they won't say something grossly offensive at a moment's notice. But it's also important for these models to have a sense of how bad text works. If you want a model that does the right thing when some sensitive subject comes up,
You wanted to understand what a bad discussion of that subject looks like. I've heard this described as sort of like, if you want an AI to not act like a Nazi, it has to know what a Nazi sounds like. It has to be fed some examples of Nazi language so that it can avoid doing that.
Right. Because someone whose job is to avoid saying things that are offensive to Jewish people, you can't perform that job if you are simply oblivious to the concept of Nazis, if you've just never heard of them. So it's important that models be trained on offensive material. It's important that they be trained on erotica, on hate speech, and so on. But the fact that they are trained on that needs to be controlled, and it needs to be addressed through fine-tuning to make them suitable for use. Right.
What is the most intractable challenge that you have come across in prompting? Like something that it seems like the model should be able to do, but the more you prompt it, it doesn't seem to work. The most intractable challenge? There are many problems that I think we know in advance cannot be done
on an LLM, or at least it would be silly to try. So calculation is a good example. If you just needed to crunch numbers for you, you can do it. So if you essentially, there's an LLM equivalent of longhand multiplication. You can have it write out the same way you would if somebody gave you a really hard math problem and your only tool to solve it is Notepad. You could write out all of the carry the one and so on, and you could really math it out in Notepad and you could multiply two 12-digit numbers together.
And you essentially have to do that to prompt the model to multiply two 12-digit numbers together. But why would you, right? The calculator is right there. And a lot of the things that excite me the most about large language models lately are getting them to not do things like that, to not attempt to solve large calculation problems themselves, to delegate that work to a computer when it's appropriate. Yeah, we had a listener write in very frustrated that ChatGPT could not give him an accurate word count on anything, which seemed like the easiest thing in the world to do.
Right. So a large language model cannot give you an accurate word count like naively, right? So if you prompt it and then just say generate a token, that is the number of words and that it will not do well on that task. But what it could do is write a program that could count the words. If it had an execution environment, it could do that very reliably.
Like if you look at ChatGPT, it was originally Code Interpreter, but they renamed it as Advanced Data Analysis. That feature is very impressive. You can even upload files to it. So you can like upload a text file and then say, tell me the number of words in this. And what it will do under the hood is write a quick script in Python that will count how many words are in this text file that you've uploaded, get the answer, and then display it to you. That's
That's like an amazing Rube Goldberg description of how to get a word count for something. Yes, it is, but it works. And one of my favorite examples, I gave it an animated GIF and I said, give this back to me, but make it a five second long MP4 video, 512 by 512 pixels and add a dramatic slow zoom in effect. And it did it. You know, I gave it the file. It gave it back to me.
So it's that there's no further conversation needed. That's amazing. Which model was that? That was ChatGPT Code Interpreter, which is now known as Advanced Data Analysis. You have to be on the premium plan to have that. Got it. I feel like some people who may be learning about the idea of a prompt engineer for the first time might think, that doesn't sound like work to me. Do you write like six prompts and think, oh, I got to take a break? What does it feel like?
Sure. I think the most clarifying answer is that it used to be harder. So in 2022, even, writing prompts took more domain knowledge. You had to know quirks like it's wrong in some sense to end a prompt in a space. There's technical reasons why that's true. It has to do with tokenization and how letters are chopped up into sequences that the model understands. But it's just a bad idea in general to ever end your prompt in a single space.
As the models have gotten better, they're getting better at, I'd say, being aware of what they can and can't do, of understanding when results are likely to be wrong, not saying completely ridiculous things quite as willingly as they used to. It's a very different mode of interaction than it used to be. And so it took more work.
prompt engineer as it was imagined at the time. I think that was a career that a lot of people foresaw that every company is going to have to hire a prompt engineer and you're going to need them to be the shaman that speaks to the model, that interprets what you want to do. That hasn't really panned out. And to be honest, I don't expect it to pan out. I think these models are getting easier to talk to. To borrow a comparison, to call yourself a prompt engineer is like calling yourself a typist. It's
It's something that in the future everyone will be doing, but nobody's going to consider it their job. Right. It reminds me a little bit of when I was a kid and the public library in my town first got the Internet. There was like a librarian, like a person who worked for the library who would basically just sit there all day and teach people how to use the Internet for the first time. It was like, here's how you, you know, here's how you dial up and here's where you go to get your email. And like, here's what email even is.
And now, obviously, that job isn't necessary because both the tools have gotten more intuitive and people have just gotten more acclimated to them. So do you worry at all that you are like that person at the library in my town who's just teaching people how to use the Internet and that a few years from now, your job will not be necessary?
I think I had a tweet about this once where I said, people always tell me that prompt engineering isn't going to be around in five years. And I'm like, damn, I spent all that money on prompt college. This wasn't even my first gig. So I'm not too worried about it. I think if prompt engineering goes away or if it morphs into something else, I'll shift into whatever more respectable thing it becomes.
But it is interesting because so many people have been looking for not only what jobs are these large language models going to displace, but what jobs will they create? And I think prompt engineer for a lot of people was the first time they heard about a job related to large language models that was new.
And it got a lot of attention earlier this year when Anthropic listed a prompt engineering job that had a salary range of between $280,000 and $375,000. So my first question is, like, is that kind of pay normal across the industry for prompt engineers? Or is that just a total outlier? Good question. For the position that Anthropic was looking for, I think what they're paying seems very reasonable to me, if not low. But...
I also would say that there's very few positions like that. The people who work full-time on prompt engineering tend to be much closer to the model providers themselves. They tend to be people who are maybe in another world could be described as solutions engineers or working with customers and helping figure out how they can best prompt the model to have their tasks done well. But I think the whole profession is in some way an outlier. You're talking about a fairly small pool of people.
My last question for you is sort of an off-ball one, or an oddball one. It's a bit of an off-ball, as we call it here in our large language model. Yeah, I haven't been fine-tuned this morning yet. Riley, you have probably spent more time talking with large language models than almost any human on Earth, if I had to guess. You're probably in the 0.1% of that.
Do you ever try prompt engineering techniques on your friends? Has it given you any ideas for real-world interactions where you're like, take a deep breath and then try explaining that workplace drama again?
I worry sometimes that I gravitate towards large language models because I dislike talking to people. That people have these consequences, right? If you're rude to them, like if you say, hey, let me try it again this other way. They don't like when you say stop generating in the middle of a conversation. Right. Like if you say, shut up, let me try it again. Like that doesn't go so well in the real world.
I would like to think that my style of prompting is not infecting my style of interacting with normal people because I'd hate to become that person. Yeah, well, you did threaten to kill an innocent man if we kept you over time on this podcast. So we will obey that order. Thank you, Riley, for coming. It's really good to talk to you. Thanks so much, Riley. Thank you so much. Take care. Bye.
Indeed believes that better work begins with better hiring. So working at the forefront of AI technology and machine learning, Indeed continues to innovate with its matching and hiring platform to help employers find the people with the skills they need faster. True to Indeed's mission to make hiring simpler, faster, and more human, these efforts allow hiring managers to spend less time searching and more time doing what they do best, making real human connections with great new potential hires. Learn more at indeed.com slash hire.
Hard Fork is produced by Davis Land and Rachel Cohn. We're edited by Jen Poyant. This episode was fact-checked by Andrea Lopez Cruzado. Today's show was engineered by Alyssa Moxley. Original music by Alicia Butte-Yutube, Marion Lozano, Sophia Landman, and Dan Powell. Special thanks to Paula Schumann, Queering Tam, Nell Gologly, Kate Lopresti, Jeffrey Miranda, and Ryan Manning.
You can email us at hardfork at nytimes.com. Maybe tell us a story. See if you can be more interesting than ChatGPT. Have a question or need how-to advice? Just ask Meta AI.
Whether you need to summarize your class notes or want to create a recipe with the ingredients you already have in your fridge, Meta AI has the answers. You can also research topics, explore interests, and so much more. It's the most advanced AI at your fingertips. Expand your world with Meta AI. Now on Instagram, WhatsApp, Facebook, and Messenger.