The Future of AI: Predictions and Realities
18:14 Share

Welcome to the AI chat podcast today. On the podcast, I wants to talk about a really interesting concept that might be throwing a serious ranch into OpenAI and throw up picking a lot of the other big players in the A I. market. That is a fact that right now, there seems to be a limit that a lot of these companies are hitting when IT comes to how fast they can actually improve their AI models. And the big thing that people are talking about here specifically is right now we went from you tb t three to five with a really big jump.

People are concerned, and inside the company are having some alarm bells, that they jump from four to what a lot of people are calling charge to beauty five, which theyve actually change the naming convention, probably to avoid this issue a little bit, they are call in yan. There are warned that this is not a significant eva jump. There's some people inside the company, and they are actually having to turn to other solutions to make these models Better because of data and compute constraint.

So today on the podcast will be breaking down exactly what's going on in the AI landscape where everything sits uh with some of these issues and how people are trying to get over some of these um hurdles or road blocks in compute where seen them really bold predictions by uh um opening eyes, C E O sam melt man and also from the C E O anthropic. So will touch on those. But there are some really interesting stuff.

Let's get into IT. Before we get into that, I wanted to mention if you haven't already and you're interested in making money from A I tools, I would love to have you as a member of the A I hustle school community. This is a place where every single week I record an exclusive piece of content that don't put out anywhere else, and I cover how to use A I tools.

How am currently making money and growing and scaling my business. Um there's over two hundred members that are from, you know they've started a hundred million dollar companies and summer starting now where a new companies. So I really wide range of perspectives um but a really incredible community to essentially learn how to grow and scale your business with A I and how to make money with A I code.

So if you're interested in this, you can check out a link in the description is nineteen dollars month will increase the Price in the future. Um but for now, you can look at that Price and it'll be raised on you and it's a great community. I'd love to have you as a part of IT.

Alright, let's get into the episode. So a big part of this actually came from an article that was broken by the information. The information always has kind of these juiced, but the interview some people inside the company and essentially found out some interesting information. So the big thing here is the chAllenge um right now, the increase in quality of you know this next flag model O I an or for some people called GPT five, is not going to be as big as the jump in the last two.

Because of this, the entire industry, not just opening eyes of a lot of other players, including anthropic, are shifting their efforts to essentially focus on different ways they can improve these AI models, not just the data that goes into IT, not just how much compute, but actually software. There's even a really pessimistic opinion that's held by a mark eaker berg. So it's definitely an interesting time to be in the industry.

But mark zg, he said that in a worst case scenario, there's he's like there there's still like lots of room to build consumer and enterprise apps in different products on top of the current technology even if IT doesn't improve. okay. This I for one I hearing this from, you know one of the companies matter who had is you know one of the top five liters in AI has come up with really big cutting edge open source AI tls.

This is not super optimistic news for me is like don't worry, guys, like even if we don't improve IT, there's lots of like apps we can build their super useful. Yes, there's lots of apps can we built that are super useful. People are going to make those.

But like we really want the underline technology to get Better. And it's interesting because at the same time, we're having in simulate men say, you know we're going to achieve agi in twenty twenty six, I believe twenty twenty five. Um so anyways, he's got some really bold predictions and his predictions are essentially based off of the rate of improvement that we're currently seen.

Is like look if you look at ah you know the if you look at the division model to the o three model, to the o four model or you know to GPT four, like these are massive jumps in improvement. And if we can keep this up, we're in age by you know next year, the year after. And I think that the C E O anthropic was a little bit more um concern, said you know along the same lines that twenty twenty six or twenty twenty seven he expected us to achieve A G I which you know people love different um they have different kind of how they classify that.

There is a couple of interesting things though that I think go into this. A lot of people are really excited and hyped sammons the first to say that he believes is gonna take ten thousand days which I think is you know a funny wave saying but he thinks ten thousand days were not like super intelligence um and you know you always always post on twitter like crazy thing which is ten thousand days away from super intelligence it's all kind of extra plating from the same thing. It's interesting because they would appear they have a different definition for artificial general intelligence and a super intelligence.

Super intelligence is something, you know, they hope to achieve this, you know, knows everything you can do, everything you can predict. Everything is insane. But what they are focusing now is a general intelligence, which is generally smarter than A P, H, D student and human and all tasks, right? That's what a is.

The reason the definition of the agi is important and achievement IT is important for sam altman, man, is because as soon as he achieves A G. I, he's out of the deal that he has a microsoft um where they get access to the A M models and they potentially could just walk back. Now I bet this can be lot of legal battles around how exactly that plays out and if they can actually close file as agia.

But anyway, that is incentive for saying he's gone to achieve the next year, the year after. And if he's saying he's gone, achieve I in the C, E, O. And throw out my cast also say, hey, look, we're going achieve agi too because they don't want to be like h, we're not going to do IT for five years and make of an open, I look like the winner.

So now all the said and everyone's got to say they can achieve IT. And I think what's gonna en is essentially just going to lower the expected of what we're calling A G S. Whatever achieve in the years about the we reach at the things pretty good.

It's like, yeah, maybe it's not perfect, but anyways, that's my that's my take on why they have those bolder predictions while IT seems like their they're struggling. So some researchers at the company at opening, I right now believe all yan, this is the latest model is not reliably Better then it's previous version, which is GPT four. This is not good news um if it's accurate and that's just for certain task.

So according to this employee that spoke to the information anonymous ly um ryan is Better at different language tasks. So that's good. But IT might not actually outperform the last model that they had when IT comes to coding and there a number of other tasks.

But this is really important. Code is a big A A big deal because this has a lot of logic and reasoning that's needed for this. Um so definitely this could be a problem um especially when you consider the fact that oh, ryan is going to probably be more expensive for open eye to run in its data centers compared to what IT currently has on the market. So it's going to be charging more money and maybe it's Better some languages es, but it's not Better at coding or or everything. Um this is going to be an issue that are going to have to try to solve.

So right now with all ryan, um this whole situation is really going to it's onna test the the assumption of the whole AI industry, which we're all looking at right now, which is the scaling laws and that is that l ms are gone to continue to improve at the sam Epace a s l ong a s t hey h ave m ore d ata l earn f rom a nd m ore c omputing p ower t o e ssentially f acilitate t he t raining p rocess. And this is kind of what saltman and the CEO of anthropic have both been saying when they're talking about when we're getting A G I. So they're really publicly endorsing ing this um kind of scaling law idea.

And so this could be a problem if ryan comes out and IT essentially has less improvements, a less of a jump, all of the same mtts. Um the timeline and direction for a tuv agi IT gets like lop sites like well, if we dimensions returns, it's can take us way longer to achieve something that's actually that good. So in response to all of these different chAllenges, the AI industry essentially is shifting.

How there addressing the scaling law, the efforts you have mark A B saying that worse case, eny, we can still build cool tools on top of IT open a ee is actually baking more you know co writing capabilities into their model um because they're trying to fight off anthropoids is a serious threat and essentially they're just developing software that can that can do a lot of what they need. And anthropic right now is even going so far as creating sort they can take over your computer and complete venture tasks. You so they're trying to make there are AI models more useful, but it's not because the AI model itself is necessary getting smarter.

It's just the adding extra software functionalities that make them more useful to a person, which is fantastic. But you also need these underline like we need the models to keep getting Better and Better because anyone can build the software. Um but you know some of these guys, uh, there is only so many people that have enough resources, talent and money to make the underlying models Better.

That's really what we want them to focus on. So what's interesting is everyone's focusing on agi, everyone's focusing on agents, which is kind of what the the next step that they're they're all taking. Um there's a researcher of OpenAI who is no one Brown and he gave a um telex conference last month. Talking about essentially that these more advanced models might actually become financially unfeasible to develop. So there might come a point when IT doesn't even make sense to develop.

And this body said, quote, after all, are we really going to train models that cost hundreds of billions of dollars or trillions of dollars at some point in the scaling paradigm breaks down? So this is also interesting, right? Because they keeps saying like a look with more data, more compute to make these things Better, Better.

But like at some point, is that worth spending a trillion dollars to train the latest model? Like are we getting bang for our bus? Is there an actual realistic trade off here? So I think these are really fascinating questions.

I'm sure in video is happy with all of them because a trillion dollars sounds probably fantastic to them, but a lot of that goes into power. Another like another, costs as well. But you, I would say, expecting video to keep growing and benefiting if this is really where where the industry goes.

okay. So open N A. I has done a lot of work with their ori and model.

Lot of people are testing IT, but right now, they have their kind of testing and safety that they are doing right now. It's quite intensive and they're still going going into IT. And yeah, do a lot of to work on that. So some people also speculate that we are going to be hitting a data wall. This is something a lot people talk about.

So this is one reason that they believe we're going to see a slowdown in the progress of these different models um is because essentially, we're getting the less and less access like the pool of really high quality text and other data that we can give to these a models blindness. We're getting less, less. But with you so much of that, there's only so much you can create.

Now some people say the solution is synthetic data. There's a much of uh you know arguments about that back and forth. So this is uh this is an interesting topic, but it's no double that.

New data sets seem to be uh in short supply at the moment. So in the last few years, l ms really just use all of the data and text that were on websites, books and anything else has been put on to the internet. Um but yeah course that's all been used.

So I wanted to talk through um just the different steps that an A I model takes. So this is essentially the training process, the testing process that these elements go through before they are actually released. Just to give you an idea of like where oh, ryans at more than these other models are at and where we're kind of moving into the future.

So the first step is set up, that is the data collection and the data preprocessing. So cleaning the data, making shirts actually usable by these A M models. Next we go to pre training. So um retraining happens. There is continuous evaluation going on.

And then we go to the evaluation, which is essentially evaluating after the retraining of that model has done, we then moved to post training, which is essentially introducing a bunch of new data. Um after collecting and kind of preprocessing happens, you're find tuning the models. Then you do reinforcement learning based on um kind of human evaluation.

So people are testing the models coming up with like feedback on things that needs to do Better or change that really get the final une in the reinforcement. You then do a pre release, which is essentially what's happening with the iran model right now where you have a pretty decent model, but people have to do safety and that kind of know testing and make sure everything is good and then we get the launch. So we're getting close on the latest orion model, right? And so when people are giving their speculations on how good IT is, it's already been through all of these steps.

It's just on the pre release right now was just on the the safety testy and and some of the further valuations before they actually do the launch. So we're quite advanced in this. okay.

So one thing that I think is really interesting um is the fact that we have been horwitz. So one of the CEO h one of the founders of a six, one of the biggest capital finds who gave a really interesting quote in the youtube recently. He said we're increasing in the number of graphic processing units used to train A I at the same rate, but we're not getting the intelligent improvements at all out of IT.

He didn't really elaborate on IT, but IT would appear to be from what he saying that they're actually giving IT more gp user given IT more processing power um and the intelligence is an actually improve. So this is some people are concerned about this and it's going to be interesting to see what happens. So what is the other solution we talk about the solution of, you know adding new software that can kind of do things.

So it's bad at math. You can not add like a math software that is essentially queries and IT uses that to to dissolve some problems. It's not really the AI model that's doing IT.

It's choosing a math calculator to do stuff. Um that's a great solution. What other solutions are we talking about? The big one that you've all recently is how chat if you came out with their o one preview right now, the GPT o one.

And essentially that is um the ability to when you ask a question, IT runs that question through more compute power. So instead of just having ChatGPT give its straight up with a response, it's going to take that response, run IT through and say, k, test this for accuracy, test this for this. Make sure to fact check this, break this, break this question down.

They're asking to seven steps. Complete the seven steps now put those steps, seven steps back together. Now condensate, they're doing all this stuff in the back end where they essentially have like a really elaborate ate prompt, really working your question out and a much more a laboratory where they do um you know the essentially work through a thought process.

So this is a fantastic uh tool that is getting IT Better but IT cost more money. Sop Brown, who is doing that whole telex AI talk recently, I was talking about this and said, quote, this opens up a completely new dimension for scaling um spending a penny per query to ten cents per query. Um so essentially researchers can improve the model responses by just spending more and essentially putting IT through more more rounds.

So sam altman also has talked about the importance of open eyes reasoning models um which is kind of this own previous which can be combined with their l ms meaning it's actually not um having to retrain a new model to make a Better. They're just able to use the same models to get the Better results by using more processing. So someone said, quote, I hope reasoning will lock a lot of things that we've been waiting years to do. The ability for models like this to, for example, contribute new science, help write a lot more very difficile code he was talking to developed his bacob tober um was going over all of this. We're seen that with more time the act and be able to essentially run the prompts and the outputs through more AI models and giving them more time in compute actually does increase accuracy, which is great um but it's not perfect.

The one thing that I will say that is great with all of this is just the fact that is just the fact that, well, we might be hitting some sort of platos on some of the models, be able to run things like, you know, run the model, run this prompt through ten different things and make sure to pick the best in and refine IT with each one, and go through step to the thought process and different steps like that makes you Better. And we essentially can just keep doing that over over again. Okay, take this proper run through one hundred times and now it's going to get marginally Better than ten through ten times.

So well, IT doesn't really expensive. You can get Better results from this. Now this is what we're kind of currently doing to to get around some of the limitations we have.

But I think of the end of day, we need to confront the limitations head on and say, how do we actually get the found models Better? That's the big thing without spending a trillion dollars on compute. A lot of that is making the model training more efficient, getting energy cost down.

There's a lot of things that we need to do to essentially unlock this and a lot of road blocks that we have to get through. But it's a very fascinating time to be N A I. And it's it's a serious problem that people are um grappling with, which is that the AI models appear to be having a slowdown of improvement just based off of more computer and more data.

Um and so we're going to have to come with solutions for this so exciting time to be alive. I don't think that this means know they're permanently stuck in limbo or a platov, but we're going have to come with new creative ways to grow. It's not just the same old, same old women of last two years.

We now need to come up with new solutions. So definitely exciting. Keep you up today on everything going on in the A I industry and everything that i'm seen here. So again, if you're interested in making money with A I tools, make sure to go check out the A I hustle school community and incredible place, and also make sure to go and get on the A I box weight less.

If you're interested, i'm in building your own A I tools in using our playground whats you access tons of different models, offer one a description cost, so you got A I box to A I links also in the description to get on our weight list for that which be launching surely. Thanks so much for tune to the podcast today. Hope you all have a fantastic rest of your day, and I will catch you next time.