Deep Dive
Why has OpenAI delayed the launch of GPT-5?
The delay is due to several challenges, including the need for more data, the complexity of model design, and the risk of synthetic data causing model collapse. OpenAI is also grappling with defining what GPT-5 should achieve, as current evaluations rely heavily on gut feelings rather than clear metrics.
What are the key challenges OpenAI faces in developing GPT-5?
The main challenges include a data gap, the need for synthetic data, and the difficulty in defining the model's capabilities. OpenAI is also hiring specialized talent to help solve complex problems and feed that knowledge into the model.
How much does it cost to train a model like GPT-4 or GPT-5?
Training GPT-4 cost over $100 million, and future models like GPT-5 are expected to exceed $1 billion in costs. The expense is primarily due to the computational resources required for such large-scale AI models.
What is synthetic data, and why is it important for GPT-5?
Synthetic data is data generated by AI, which is then fed back into the model to help it grow. It’s crucial for GPT-5 because there’s a significant gap between the projected size of the model and the available real-world data. However, synthetic data carries risks, such as causing model collapse if the wrong type of data is introduced.
What are the expectations for GPT-5's capabilities?
Expectations are high, ranging from automating tasks like booking flights and appointments to predicting user needs and reducing mistakes. The model is expected to 'wow' users with its capabilities, but there’s no clear consensus on what exactly GPT-5 should achieve.
How does the delay in GPT-5's launch affect OpenAI's business?
The delay could impact OpenAI’s competitive position, as the company has set high expectations for GPT-5 as a significant step forward. If the model fails to meet these expectations, it’s unclear how the business will respond, though there’s still potential for incremental improvements.
What are some creative digital gift ideas for the holidays?
Creative digital gifts include subscriptions to apps like Time Shifter for travelers, Babbel for language learning, and password managers. One-time fee apps like Crouton Plus for cooking and Animal Crossing Pocket Camp for gaming are also popular choices.
How can digital gifts be made more personal?
Digital gifts can be made more personal by printing out gift cards, wrapping them, and including a related accessory. For example, a yoga app subscription could come with a yoga towel, or a Netflix gift card could be paired with popcorn.
- Digital gifts can be as thoughtful as traditional presents.
- Examples include Netflix subscriptions, iCloud storage, language learning apps, and password managers.
- Presentation is key; wrapping the digital gift card and adding a small, related physical item enhances the experience.
Shownotes Transcript
Amazon Q Business is the generative AI assistant from AWS, because business can be slow, like wading through mud. But Amazon Q helps streamline work, so tasks like summarizing monthly results can be done in no time. Learn what Amazon Q Business can do for you at aws.com slash learn more. Welcome to Tech News Briefing. It's Monday, December 23rd. I'm Danny Lewis for The Wall Street Journal.
Christmas is just two days away, so if you're still shopping for presents, our personal tech columnist has ideas for digital gifts and how to make them more personal than the dreaded gift card. And later in the show, OpenAI is working on its next highly anticipated artificial intelligence model, GPT-5. But the journal reports that delays have pushed the launch back, and the development cost is skyrocketing.
Our reporter Deepa Sitharaman joins us to explain why and what the delays could mean for OpenAI's business. But first, gift cards have a reputation for being the present of last resort. But with more and more of our lives taking place on smartphones and online, our personal tech columnist Nicole Nguyen says that's no longer the case. And there are ways to make digital gifts just as personal and thoughtful as any other present.
Nicole, when we're talking about giving friends and family digital gifts, aren't we really just talking about, you know, the dreaded gift cards? The answer is yes. But gift cards really don't have to feel impersonal. And if you do your research and make sure you're getting the right kind of subscription that's actually helpful, that's actually useful, maybe you're chipping into one of the many digital subscriptions that they're probably paying for every single month.
It could feel as good as a coveted Lego set unwrapping a gift card. So what kinds of digital gifts would make thoughtful holiday presents? Okay, well, listen, we spend nearly $1,000 a year on virtual goods by some estimates. That's a lot of money. And so you won't go wrong with the staples. So that's Netflix, Netflix.
Apple credit for iCloud storage for the many thousands of photos that we're storing every year or for the Google App Store. But if you want to go more creative, for frequent travelers, there's a very cool app called Time Shifter that's a sleep guide and it can help prevent jet lag if you are traveling abroad.
If your gifty is interested in learning a different language, Babbel is very cool. There are bite-sized lessons for 14 different languages. My favorite digital gift to give, which I give my loved ones every year, is a subscription to a password manager. I know that's not very sexy, but I do offer my services, which is the most valuable part of this gift.
You get a year to one password, and then I help you change all of your bad passwords and set up your password manager on all of your devices. There is no price on that gift. One thing about the subscriptions is they keep coming up.
You know, if you gift someone a subscription and they end up liking it, aren't you just sticking them with another payment down the line? Yes, you are. And so if you don't want to give them the gift of a recurring monthly fee, you can give them a paid app that has just a one-time fee. And it's actually very easy to gift these apps from the App Store app on iPhones. Some of our favorites include Crouton Plus, which
which has a ton of neat cooking tricks. You can take a picture of any cookbook recipe, and with this app, you can scale that recipe to your desired party size. Animal Crossing Pocket Camp Complete is one of Nintendo's newest games. It's an adorable life simulator that's 10 bucks. And a gift that I have given in the past is called Forest, which rewards your productivity. You open Forest up on your app, and the longer...
it's up and so you're not distracted by other apps like TikTok, the longer you focus, the lusher your digital garden. So as thoughtful as these digital gifts can be, it can still feel impersonal to just send someone an email with a download code or something like that. How do you make them feel more special?
I talked to a lot of etiquette experts about this, and they said you should print out your gift card or buy a physical gift card and wrap it like you would a new pair of skates or a new bike and give a little a
a fun little side accessory along with your gift card. So if you're gifting a yoga app, then you should include a yoga towel. If you're getting online French lessons, maybe include a fresh baguette. Or if you're gifting a year worth of Netflix or Disney Plus or Max, how about a little tub of popcorn? WSJ personal tech columnist Nicole Nguyen. Coming up, there's a lot of hype around OpenAI's long-delayed update to the AI model that powers ChatGPT.
When might it arrive? And what does it need to do to meet expectations? That's after the break. Amazon Q Business is the new generative AI assistant from AWS because many tasks can make business slow, as if wading through mud.
Help? Luckily, there's a faster, easier, less messy choice. Amazon Q can securely understand your business data and use that knowledge to streamline tasks. Now you can summarize quarterly results or do complex analysis in no time. Q got this. Learn what Amazon Q business can do for you at aws.com slash learn more. OpenAI has been working on a new update to the AI model that powers its popular chatbot, ChatGPT.
The company has been hyping up the long-awaited update called GPT-5 and codenamed Orion, but it has faced significant delays with only a smaller upgrade called GPT-4.0 released this year.
And estimates suggest a six-month training run for a new model can run about half a billion dollars in computing costs alone. OpenAI has conducted at least two training runs. WSJ tech reporter Deepa Sitharaman has been following this story. She joins us now to explain what's going on. And a quick note before we get into it. News Corp, owner of the Wall Street Journal, has a content licensing partnership with OpenAI.
So, Deepa, what do we know about GPT-5? How is it supposed to be different from GPT-4.0?
It depends on who you're asking. Some people think that it will be magic. And then a lot of other people just want this entire system to be able to actually do things for people like book flights or book appointments and restaurant reservations and maybe even predict what you, the user, want and make fewer mistakes.
So it sort of runs the gamut. And there are very high expectations for this model to do any of these things, but just to generally wow. So why is this new model so late? There are a lot of really complex, different reasons for that. First, we have to acknowledge that these systems, they're very, very difficult.
And OpenAI has some of the smartest people in the world working on that issue. But what's happening is that until now, there's been one very powerful idea that has served as the engine for growth within the AI community. And that is scale. Scale matters. There are more nuances to this, but essentially the idea is to chuck all the data you have into
into the model and somehow the model just becomes more capable.
That method, though, only works if you have the data. So we're in a situation right now. I mean, there's a lot of data on the Internet, but there's a big difference between the texts between me and my mom and public data. Everything good has pretty much been scraped. And if you're doubling the size of these models, you have to increase the amount of data per
disproportionately, and currently there's a really big gap between what the projected size of the model is and the available data. We should note that OpenAI and Microsoft declined to comment for this story. Deepa, what has OpenAI said about this delay?
Currently, they're not saying anything. Our understanding from talking to sources that are around the issue is that Microsoft executives there really thought that it would be ready by the spring of 2024. And now we're talking in December 2024, and it's still not out. And that's because they've encountered a lot of different challenges.
First, they need to decide the design of the model. You want something that can sustain such a gigantic amount of data. Then there's this other challenge around data. Like I said, you need more data. And so OpenAI and the researchers have had to figure out what do you do? How do you fill that data gap? And they've approached it in a couple of different ways. And
One key way is by making it. This is called synthetic data. It's when AI makes data and then you give it back to the AI in hopes of it growing. It's also a source of risk if the wrong type of data is introduced. A lot of research that's shown that certain types of synthetic data actually cause the model to fully deteriorate. They call it model collapse.
So it's not without risk to do something like that. And then they are also hiring people and not just, you know, anybody. They're looking for software engineers or mathematicians and in some cases, people like theoretical physicists that can help solve problems and then explain their logic, like the way they thought through the problem. And that material is then fed into the model and that material
Data could really help the model understand, hey, this is how this person solved the problem. Maybe I can also come up with a middle way or adopt one of these two approaches and approach new problems with this strategy in mind. And then there is this other challenge, which is the fact that
No one knows what GPT-5 is supposed to do or be like. There are certain tests. Right now, you've got...
evaluations. But fundamentally, it's coding and math problems. Those things are easy to evaluate because code works or it doesn't. And math, it's the answer's right or it's not. Obviously, there are exceptions when you get to very high levels of both of these subjects. But how do people at OpenAI gauge whether or not what they're producing is worthy of the GPT-5 moniker?
And for a lot of them, outside of the evaluations for those subjects and a few others, it's gut, your gut feeling. What one person described it to me was it's vibes, mostly vibes. And Deepa, you also report that cost is a factor here. OpenAI CEO Sam Altman has said training GPT-4 cost more than $100 million.
Future AI models are expected to push past $1 billion. So what happens if OpenAI is not able to release better models on this schedule that they've developed?
How does this delay affect OpenAI's business? They will always be able to push the model a little bit further. There is still a lot of low-hanging fruit around things like reasoning. But, you know, when executives at OpenAI describe it as superintelligence or a significant step forward, that sets the bar really high. And so the company is able to push forward and do interesting things. Is it going to be that thing?
That's the open question. And it isn't clear how the business will respond if it fails at creating the next order of magnitude. Deepa Sitharaman covers artificial intelligence for The Wall Street Journal. And that's it for Tech News Briefing. Today's show was produced by Julie Chang with supervising producer Catherine Millsap. I'm Danny Lewis for The Wall Street Journal.
Starting tomorrow, we're kicking off a series about the biggest trends, people, and companies our tech reporters and columnists will be following in 2025. We'll be off on Wednesday for Christmas and back on Thursday with a new episode of Tech News Briefing. And we'll be back this afternoon with TNB Tech Minute. Thanks for listening. Amazon Q Business is the new generative AI assistant from AWS because many tasks can make business slow, as if wading through mud.
Help? Luckily, there's a faster, easier, less messy choice. Amazon Q can securely understand your business data and use that knowledge to streamline tasks. Now you can summarize quarterly results or do complex analysis in no time. Q got this. Learn what Amazon Q business can do for you at aws.com slash learn more.