#381 – Chris Lattner: Future of Programming and AI
03:38:29 Share

The following is a conversation with Chris ladner, his third time in this podcast. As i've said many times before, he's one of the most brilliant engineers and modern computing, having created alem compiler infrastructure project that claim compiler, the swift programing language, a lot of key contributions. The TensorFlow ing tp uses part of google, he served as vice president of autopilot, often are tesla, was a software elevator and leader at apple, and now he co created a new full stack A I infrastructure for distributed training, influence and deployment on all kinds of hardware called modeler, and a new programing language called module that is a super set of python, giving you all the use ability of python.

But with the performance of C, C plus plus, in many cases, mojo code has demonstrated over thirty thousand x speed up over python if you love machine learning, if you love python, you should definitely give mojo try this programing language, this new AI framework and infrastructure. And this conversation quiz is mind blowing. I love IT.

IT gets pretty technical time, so I hope you hang on for the ride. And now a quick few second mention of sponsor checked him out in the description is the best way to support this podcast. But I heard for health numerous world's harder state science.

Turn ament and insight tracker for tracking your biological data. Choose one of my friends also, if you want to work with our team, our amazing team are always hiring, could elect governments, dock, house, lash hiring. And now onto the full ad reads, as always, no ads in the middle.

I try to make this interesting, but if you skip them, if you must, my friends, please still check out the sponsors and enjoy their stuff. Maybe you will too. This show is brought you by I erb, a platform, a website, a place we can go and get high quality selected just for you.

Health and wellness products for great value and expensive, affordable. I get visual over, there is one of the main supplements have taken for a long, long, long time in pill form, makes me feel like i'm oiling the machine, that is the human body and the human mind. Even just saying that makes me wonder what is the power of the places al effect in all of this? I actually a big believer in the power of the human mind, coupled with the effectiveness of a medication and supplements and nutrition and died and exercise all of IT.

If you couple the belief that the thing will work with stuff that actually works, it's a supercharge is the mind. There's something about the mind allowing the thing to work. IT may be the belief that IT works, reduces stress and has kind of secondary internal effects that you can even comprehend on the entire of the biological system, that is, the human body.

It's so facility and is so difficult to do good studies on that. Because the whole point is you want to study the effect of the entire of the lifestyle, diet decisions you make, and the entirety of the human organism, the billions of organisms that make up a single organism. That is, you anyway get twenty percent off with promotion de legs when you go to eye herb that com slash legs.

This shows also brought you by numerous a hedge fund that uses A I A machine learning to make investment decisions. Equate a tournament, a chAllenge for all machine learning gurls to come and to a compete against each other to build the best predictive models for financial markets. The stakes are high.

This is the kind of problems in the machine learning space that I really care about, real world problems with high stakes, not toy problems, not imagine now image and all that kind of stuff is good for expLoring little ideas, the nuances of architectures, training procedures of cool, little ideas of the entirety of the pipeline of how to do machine learning, or for education purposes. But if you want to really developed ideas that work in the real world, you should be working on real world data, with the stakes are high, and this is probably one of the hardest tern ament. From machine learning in the world, head over to numerous slack sign up for tournament and hou machine learning skills that numerous consulting h legs for chance play against me and win a share of determined Price pool.

This shows also about to buy inside tracker service are used to try biological data has always saying the entirety of the biological organism, that is, you, the individual cells, all the life force, that makes up the details of the cells in the proteins and the blood and the organs and the individual systems that are interconnected to each other in these incredible, complex ways that are all is synchy ously. Communicating with each other through chemistry, through physics, through electrical signals, through mechanical signals, through whatever the house signals that I can you and find the right words for. It's an incredible system.

How is this possible? That distinct works at all? I'm looking out into the distance knows. I say these words, and my visual cortex is processing all of them somehow makes sense of all is incredible. Anyway, measuring data and this incredibly organism is good.

And companies that allow you to measure IT in the order to make lifestyle decisions is obviously the future. That's why I supports that tracker. You can get special savings for a limited time when you go to inside tracker that come slash legs. This is the last way in podcast to support IT. Please check out our sponsors in the description and now your friends, here's Price winner.

It's been I think, two years since we last talked in that time you somehow went and go created a new programming language called mojo. So it's optimized for AI. It's a super set of python. Let's look at the big picture. What is division ah for .

mojo for mojo? Well, so I made the gift to zoom out. So i've been working on a lot of related technologies for many, many years.

So i've work on a lot of things and mobile and servers and things like this. But the world's changing and what's happened with the eyes? We have new G, P, S. And new machine learning accelerators and other access and things like that, that make a ago row fast, a google work on T, P, S, as one of the biggest water scale deployed systems that exist here.

And really, what you see is if you look across all of the things are happening, the industry, there's this a new compute platform coming and it's not just about C P S or G P S or T P S or N P S or I P S or whatever all the P S, right, it's about how do we programme these things s right. And so software folks like us, right, IT doesn't really good if there is an amazing hardware that we can use. And one of the things that you find out really quick is that having the theoretical capability of programing something, and then having the world's power and the innovation of all the, all the smart people in the world get unleashed on something can be quite different.

And so really, where module came from was starting from a problem of, we need to be able to take machine learning, take the infrastructure, is that make IT way more accessible, way more using, way more understandable. Normal people and researchers and other folks are not themselves like experts and G, P, S. And things like this. And then through that journey, we realized, he, we need some text, this, we need to do problem language.

So one, one of the the main features of the language, I say so fully just, is that IT allows you to have the file extension to be A I moi or the fire mog, which is one of the first emogene used as a first tension i've ever seen in my life. And then you ask ourself the question, why in the twenty first century I were not using uncoated for file extensions? Is that means an epic decision? I think clearly the most important decision mean most. But you could also just use M O, J O as the first.

So OK so take a step back and come, like you think that the world's ready for this. This is a big moment in the world.

right? This is released to the world.

This is innovation.

I mean, IT really is kind of brilliant. Emerges is such a big part of our daily lives. Why isn't not in programme well?

And like you take a set back and look at what file extensions are right? They're basically meddle a right. And so why are we be spending all the stream space on them and all the stuff also? You know, you have them stacked next text files and P, D, A file.

And what roles like you can do something cool. You want to stand out, right? Emotions are colorful. They're very, they're beautiful.

Has been the response so far for more is is their support unlike windows on Operating system in displaying like file .

explore yeah the one promise in is the git doesn't escape IT right? And so he thinks that the firm og is on principle and that like prints out weird hacks things if use the command line, get to. But everything else as fas somewhere works fine. And I I have faith that get can be improved.

So b is fine.

Get up fine. Get fine. Visual code windows, like all the stuff, totally ready, because people will have internationalized in their Normal part of their path.

So this is just, I take the next year right somewhere between, oh, that makes sense. cool. I like these things to, oh my god, you're killing my baby.

Like, what are you talking about? This can never be like, I can never happen. How i'm going to type this like all these things.

And so this is something where I think that the world will get there. We don't have to bet the whole farm on this. I think we can provide both pass, but I think IT would be great.

Um when can we have a modest as part of the code?

I wondered. Uh yes, so I am lost like we just provide that. So um I think that we have partial support for that.

Why not fully done yet? But but yeah you can you can do that, for example, in swift, you can do that for sure. So example we give give that apple was the the dog care yeah so that's a classical mac heritage thing.

And so use the dog in the cambodge together and that could be name. But of course the internet went made pilot pop for everything. Yeah so you know, if you are name your function pilot pop, then you can totally go to town and see how could review.

okay. So let me just ask a bunch of annual questions. But so is mojo primarily designed for a?

Or is IT a gentle urp? And so A I is driving law, the requirements. And so um module er is building in designing and driving module ford is not because it's an interesting project theoretically to build is because we need IT.

So what module we're really tackling the AI infrastructure, the landscape and the big problems in A I, the reasons that is so divulge to use in scale and adopt and deploying like all these big problems in A H and so we're coming out from that perspective. Now when you do that, when you started tackling these problems, you realize that the um solution to these problems isn't actually A I specific solution. And so what we're doing, this rebuilding motor would be a fully general program language that means that you can um obviously tackle G P S and C P S and like these A I things.

But it's also a really great way to build nui and other things like that. Or you know, just if you look at what many python libraries are today, often there are a layer of python for the and they end up being seen and sea bus plus could underneath them. That very true. And A I that children lots of other demands as well. And so any time you see this pattern, that's an opportunity motor to help simplify the world and help people .

have one thing to optimize through simplification by having one thing. So you mention modeler modules. The programming language module is the whole software stack.

So just over year ago, we started this company called modeler. Yeah, what modelers about is about taking A I and up leveling IT into the next generation, right? And so if you take that back, what's gone on in last five, six, seven, eight years is that we've had things like ten of and py orge.

These other systems come in, you use them, you know this. And what happened is these things have grown like crazy, and they get tons of users, is in production deployment scenario, is being used to power so many systems. I am all around us now used be controversial years ago, but now is the thing um but the chAllenges these systems is they haven't always been um thought out with current demands in mind.

So you think about IT when where were elms eight years ago? They didn't exist, right? A I changed so much.

And a lot of what people are doing today are very different than when these systems were built. And meanwhile, the harper side of this has gone n into a huge mess. There's tons of new chips and cells and every every big companies announcing into chip every day feels like.

And so between that, you have like this moving system on one side, moving system on the other side, and just turns into the jgc mess, which makes a very difficult for people to actually use A I, particularly in production deployment ts and energies s and what module sh doing were helping build out that soft stack helps solve those problems, so then people can be more productive and get more AI research international. Now what mojo does is, is a really, really, really important piece of that. And so that is a part of that engine and part of technology that allows us to solve these problems.

So mojo is a programming language that allows you to do the high level programing, the low level programing. They do all kinds of programing in that spectrum that gets you closing closer to the hardware.

So effect. So what do you love about python?

Oh, boy, where do I begin? What is love? What do I love .

about piton?

You guy know love? I know this. Yes, how to of IT is how IT feels like i'm writing natural language, english. How when I can not just write but read other people's code, somehow I can understand IT faster. It's more condensed ed than other languages like ones are really familiar with like the and see uh there's a bunch of sexy little features yeah uh we've probably talk about some of them, but less comprehensions is tough like this also put and .

don't forget the the entire ecosystem of the pack is probably huge. There's always something if you wanted do anything. There's always a package.

yeah. So it's not just, uh, the ecosystem of the packages and the ecosystem of the humans that do IT. That's a really that initially dynamic because because I think something about the the usability and the ecosystem makes the thing virgos, and it's a virtuous cycle.

I think there's many things I want to that like. So I think that ml was very good for python. And so I think that tens of pyro systems embracing python really took and help python grow.

But I think that the major thing i'm relying IT is that pythons like the universal connector, right? I really helps bring together lots of different systems. You can compose them and build out larger systems without having understand how IT works. But then what is the promise by thon?

I guess you could say several things, but probably that is slow.

I think that's usually what people complain about, right? And so so I mean, other people complain about tabs and spaces versus currently races or ever. But I mean, those people are just wrong because IT is actually just Better to.

and wow.

strong words.

So actually a small danger. Let's take that.

Let's take all kinds of dangers.

I've come on like, you can give me time, decide. Listen, i've recently left e max for V S code and the kind of hate mail I had to receive, because on the way of doing that, I also said i've considered them you and I chose not to with the .

s code on deep religions.

Anyway, a tabs is an interesting design decision. And so you really written a new programing language here. Yes, IT is A A super set of python where you can make a bunch of different interesting decisions here. And you chose actually to stick with path on as. In terms of some of the .

syntax so so anyway right so um I mean you can explain this in many rational ways. I think that the invitation is beautiful, but that's not a rational explanation, right? So but I can defend IT rationally right at the first path on one has millions of programmer.

Yeah, this is huge, as everywhere wants machine learning, right? So actually, IT is the thing, right? Second all, if you look at the sea, cos, even plus code, java, whatever, swift, curly brace languages also runs reforming tools and get indented.

And so if they're not indented correctly, first I will twist your brain round, especially debugs. There's niche oria bugs that have happened across time where the the entails wrong sleeping and IT wasn't format right and so turn into issue right. And so what is a happening in modern other scale? Could basis of people run automatic four matters? So now we end up with this instance and collieries, you're going to have, you know, the notion of grouping. Why not have one thing and get rid of the clutter and have a more beautiful thing, right? I'll see you look at many these language like okay, well, you can have clearly races or you can omit them if there's one statement or you just like any of this entire world of complicated design space that objectively you don't need if you have a thon saltation.

So yeah, that looks actually see statistics and errors made because of I have many areas are made in python versus cy plus that have to do with basic formatting on that kind of stuff I would love to see.

I think it's it's probably pretty minor because once you get uh like you use VS code, I do too. So if you get VS code set up, but does the innate for your general right so you don't you know actually nice and not have to fight IT. And then what you can see is the other is telling you how you could .

work by indi is very cool. And i've ever, I don't remember having an iron python because I intend to stuff wrong. So I mean.

I think that there's, again, this is a religious thing. And so I can joke about IT. And I love, I love to kind of I realized that this is such a polar ized thing and everyone wants to argue about.

And I like poking at the bear a lot of IT, right? But but Frankly, right. Come back to the first point. Python one like it's huge.

It's an A I um it's the right thing for us like we c mos being an incredible part of the python ecosystem. We're not looking to break python or change IT or on. So we love python for IT is our view is that python just not done yet. And so if you look at you, you mentioned python being so well, there is couple of different things to go into that can talk about if you but one of them is that just doesn't have those features that you would use to do see like programme.

And so if you say, okay, well, i'm forced out of python into sea for certain news cases more than what what do you were saying? Okay, well, why? Why is that? Can we just add those features are missing from python back up to mojo and then you can have everything is great about python, all the things that you're talking about you love plus not be forced out of IT when you do something a little bit more competition al intense or weird or heart ware or whatever does you doing well?

Um a million questions I want to ask high level again is a compiled there's an interpret al language so pithy is just in time compilation what's .

what's so mojo complicate the answer does all the things so interpreted is just compiled and its specially compiled um and so this is for a variety reasons um so uh one of the things that expert and beautiful is that it's very dynamic. And because this dynamic, one of things they added is that has this powerful meter programme future.

And so if you look at something like PyTorch or TensorFlow or or um I an even a simple simple use case like you to find a class that has the plus method, right, you can overload the dunder methods like dunder ad for example, and then the plus method works on your class. And so IT has very nice and very expressive dynamic meta programme features in mojo. We want all those features come in like we don't want to break that.

We warn all the work. But the promise you can't run those super dynamic features on betta processor or on A G P U, right? Or if you could, you probably want to just because of the performance.

And so we entered this question of saying, okay, how do you get the power of this dynamic metal programing into a language that has be superficial in specific cases? And so what we did was said, okay, we will take that python has an interpret, and take the interpret and allowed to run a compel time. And so now if you get if you get compel time, metal programing.

And so this is super interesting, super powerful, because one of the big advantage you get, if you get python style expressive apps, you get the ability to have over little Operators. And if you look at what happens inside of like pitot, for example, with automatic different as you go on like all these things, they're using these really dynamic and powerful features at run time. But we can take these features and lift them so they run a compile time.

So yeah as possible matter programing time plates but it's really messy.

It's super messy. It's always um IT was accidentally I mean, different people have different interpretations. My interpretation is that IT was made accidentally powerful. IT was not designed to be turing complete, for example, but that was discovered kind of along the way accidentally.

Um and so there have been a number of languages in the space and so they usually have templates couldn't dentition could copying features of various sorts um some more modern languages or some more new neural languages, let's say like yeah they're fairly um unknown like zig for example, um says, okay, well let's take all of those types. You can run at all those things. You can do a run time and allow them to happen at compile time. And so one of the problems of c plus plus, I mean that which is one .

one of the problems of c plus plus.

every go I turn off the way they show love, have a ten enough c plus plus code to earn a liberal grinds of c plus plus but um about one of the problems with that is the metal programing system templates is just a completely different universe from the Normal runtime programme world. So you do meta programing and programing just like a different universe, different syntax, different concepts, different stuff going on.

And so again, one of her goes with modules to make things really easy to use, easy to learn. And so there's a natural stepping stone. And so as you do that, she's so okay.

Well, I have to do programing at run time. I have to do programing at compile time. Why are these different things?

How hard is that to pull out? Because that sounds to me as a fan of matter programing in six plus ve. And how hard is IT to pull that off? That sounds really, really exciting because you can do the same style programing at compile time in a runtime as really.

really expected. And so I mean, in terms of the compiler implementation details, it's hard I I won't be shy about that is super hard. IT requires I mean, what module has underneath the covers is a completely new approach to the design of the compiler itself. And so this builds on these technologies like ea are the invention that also includes other like cashing and other interpreters .

and compilers .

and super compiler. And so IT really takes the standard model of programing languages and kind of twisted and unify that with the run time model, right, which I think is really cool to me.

The value of that is that, again, many these language have metal programme features like they grow macro s or something right this right, I know your roots right um you know and this is a powerful thing, right? And so you know if you go back to list one of the most powerful things of about is that I said, the matter programme, the programme are the same, right? And so that made IT way, simpler way, more consistent way, is easy to understand this about. And I made IT more composer. So if people, the library, you can use that both at run time and computer.

which is pretty cool. Yeah, for machine learning, I think that a programing, I think we could generally say, is extremely useful. And so you you get features, I think i'll jump around. But 这是 the feature of auto tuning。 An adaptive compilation .

just blows my mind OK, so shine learning, shine a mol mol. Um it's really interesting to me because what PyTorch and what tends flown, all these frameworks are kind of pushing compute.

And to is they're pushing into like this abstract specification of a compute problem, which then it's map in how much of different ways, right? So this is why I became a matter programme problem, as you won't be able say, cool, I I have this, no, that now run with batch size a thousand, right? Do, do, do, do a mapping across batch or okay, I want to take this problem now, run across a thousand C, P, S 和 G P S， right? And so like this this problem like described the computer and then map IT and do things and transformer or are like, actually it's very profound and that's one of the things that makes machine learning systems special.

Uh may be can you described attuning and Hardy pull off mean, I guess adaptive compilation is what we're talking to us meta programming, how do you pull off order to I means that is that as performs I think that is seems like I really like you mentioned this comprehensions to me from a quick glass and mojo um which by the way have to absolutely like dive in as I realize how amazing this is I absolutely must have in A A dad looks like an incredible feature for machine learning people yeah .

so so what is order change? So take a step back. Training is a future mojo. It's not so very, very little of what we're doing is actually research. Like many, these ideas have existed in other systems and other other places.

And so we're doing we're pulling to get good ideas, remixing them and making them into a hopefully a beautiful system, right until after tuning. The observation is that turns out harbour systems, algorithms really complicated. Turns out maybe you don't actually want to know how the harvard works, right? A lot of people don't, right? And so there are lots of really smart, hurt people.

I know a lot of them, uh, where. Know everything about, okay, that the cash size is this and the number of registers is that. And if you use this White length of vector is going to be a super efficient maps directly.

And what you can do, and like all this concept of the GPU, has SMS and has a web size of whatever, right? All the stuff that goes into these things where the title size of TPU is one thousand and eight, like these, these factors, right? My belief is that most Normal people, and I love harder for people all so i'm not trying to find literally everybody in the internet um but uh most programmers actually don't want to know this stuff.

And so if you come out of from perspective of how do we allow people to build both more abstracted but also more portable code because you know, could be that the factual length changes with the cash size changes, could be that the tile size for matrix changes with the number, you know, eight one hundred verses and eight one hundred verses of volta verses and whatever GPU have different characteristics, right? A lot of the algorithms, the run and actually the same, but the premiers, these magic numbers they have filling and of being really fit numbers that an expert has go figure out. And so what idea does says, okay, well, guess what?

There's a lot of computer other, right? So instead of having humans go and really try all the things, or do a grad search, go search completed space, how would we have computers do that? So what automating does?

As you can say, hey, here's my algorithm, a matrix Operation or something like that. You can say, okay, I want to covered up into blocks. I going to do those blocks in parallel, and I want one hundred and twenty things are working on. Anyone can, this way, that way, whatever. And you can say, hey, go see which ones actually imperative Better on the system.

And then the result of that, you cash for that system. I say that.

And so come back to twisting your compiler brain, right? So not only does the compiler have an entrepreneur to do matter programing that compile that entrepreneurship, ter programing now has to actually take your code and go run up on a target machine, see which one IT likes the best, and then stitched in and then keep going.

right? So part of the compilation is machine specific.

yeah. So I mean, this is an optional feature. I you don't have to use everything, but yes, if you so one one of the things were in the quest of is ultimate performance.

Yes, right? Ultimate performance is important for a couple of reasons itself. You're an enterprise, you're looking to save cost and computer and things like this.

Ultimate performance translates to, you know fewer servers like we care about the environment. Hey, Better performance leads to more efficiency. I mean, you could joke and say like you know python's bad for the environment. So if you have moved to modules like at least ten Better just out of the box and going right um uh but but performance is also interesting because at least a Better products. And so in space machine learning, right, if you reduce the lencs of a model so the runs faster every time you query the server around the model that takes less time, then the product team can go and make the model bigger. Let's actually make IT to have a Better experience .

as customer. And so a lot of people care about that. You was specified like a budget options to try shein the code SHE is simple statement, and then you just set and forget. And no, depending what worried compiles, it'll actually be the fastest. exactly.

The beauty of this is that IT helps, you know, hope, different ways, right? If you're building often what happen as know you want to bunch soft yourself, right? You you wake up on day, say, I have an idea and go put up some code.

I get to work. I forget about IT and move on with life. I come back six months a year or two years or three years later, you does IT off and you go use IT again in new environment. And maybe your gp is different. Maybe you're running on a server instead of a laptop, maybe whatever, right? And so the problem now, you say, okay, well, if I mean, again, not everybody cares about performance, but if you do, you say, okay, well, I want to take advances of these new features.

I don't want to break the old thing though, right? And so the typical way of handling this kind of stuff before is you're talking about, since temples are you're talking about, see, with microbes, you end up up with the if depth you get, like all these weird things get later in, make the code super complicated. And then how do you test IT right? IT becomes this crazy complexity, multi dimensions, space after worry about. And you know, that just doesn't scale very well. actually.

We just jump around before I goes. Some specific features like the increasing performance here they were talking ball can be just insane. You write that model can provide a thirty five thousand x speed able Price on how do they do that?

Yeah you can even do more but uh, will get to that. So, uh so first of all, when we say that we're talking about what's called sea python, the default python, the every rate users may take python through, that's like typically the one use right? C python is an entrepreneur entrepreneurship, an extra layer of like bike codes and things like this that they have to go read, parts, interpret and make them kind of slow from that perspective.

And so one of the first things we do, we move to a compiler. And so just move to a compiler. Getting the entrepreneur is two to five, ten speed up, depending on the code. So just out of the gate, SHE is more modern techniques right now. If you do that, one of the things you can do, you can start to look at c python, start to lay out data.

And so one of the things that um c python did, and this isn't part of the python back necessarily, but this is a sets of decisions, is that if you take an image, for example, it'll IT an object because empire on everything's an object. And so they do the very logical thing of keeping the memory representation of all objects the same. So all objects have a header.

They have like payable data. What this means is every time you pass around object, you passed them on a pointer to the data. Well, this has over hood and IT turns out that modern computers don't like chasing and pointers for emotion, things like this.

That means you have to allocate the data means after reference count, which is another way of the python users to keep track of memory. And so this has a lot of overhead. And so if you say, okay, let's try get that out of the heap, out of a box, out of an election and into the registers, that's that's another so IT adds up if you .

if you're reference counting every single every single thing you create that adds up .

yeah people know people complain about the python gill, this is one of the things that hurts um that's because the reference scenting so the girl and reference scenting are very tightly or twin and is not the only thing but it's very tightly intertwined and so then you linked the things say, okay, cool. Well, modern computers they can do more than one Operation at a time. The vectors, what is a vector or a vector loves you.

Take one instead, taking one piece of data, doing an ad or motivation, and take up the next one. You can now do a four or eight or sixteen or three, two time, right? Well, python doesn't expose that because of reasons.

And so you can say, okay, well, you can adopt that. I have threads now. You have like additional things like you control memory.

And so what more lows you to do is that allow you to start vantage? All these powerful things have been built the hardware over time. And that gives the library gives a very nice features. So you can say just paralyzed studien parallel, right? So it's very uh, very powerful weapons against slowness, which is why people have been, I think, having fun like just taking code and making so fast because it's just kind of a done us to see like how fast you can get things.

Before I talk about some of the interesting self paralyzing all that first talk about, like the basics, we talked the invention, right? So this thing looks like piton. It's sexy, beautiful, like path I mentioned. Is that a typed language? So what's the role of types?

Yeah good question. So python has types and has strings, has enders, has dictionary and they all that stuff, but they all live at run time, right? And so because all these types look at runtime in python, you never you you don't have to spell them.

Python also has like this whole typing thing going on now and a lot of people use IT. Yeah, i'm not talking about that kind of a different thing. We go back to that if you want but typically the um you know you just say I I have a death and my death takes two primers on, can call them and b and a so that is great.

But what that does, that forces what's called a consistent representation. So these things have to be a pointer to an object with the object hide. And they all have to look the same.

And then when you dispatch a method, you go through all the same, different past, no matter what the the receiver, whatever that type is. So mother does allow you to have more than one kind of type. And so what that does not allow you to say, okay, cool, I have I have an object and objects behave like python us.

And so it's fully DNA ic, and that's all great for many things. Classes like that's all very powerful and very important. But if you want to say, hey, it's an image, it's three, two bits or six, four years or whatever is is a floating point value has six, four bits.

Well, then the compiler can take that. And I can use that to do way Better optimization. And turns out, again, getting over of the interactions as huge means you can get that code completion because you have um because compiler knows what the type is and so knows what Operations work on IT.

And so that's actually pretty huge. And so what mojo does allow you progressively adopt types into your program and you can start again, it's compatible with python. And so then you can add, however, my types, you wherever you want them and if you don't want deal that, you don't have to deal with that. And so of our opinions on this is, is not that types are the right thing or the wrong thing, is that they're useful thing .

with those kind of options, not strict typing, you don't have to specify type exactly. okay. So starting from the thing that pythons kind of reaching towards right now. Trying to inject types into the body.

Yeah the very different approach.

But yes, yeah was the different approach. I am actually one of the people. They have not been using types very .

much in python.

So I why did you say I just well because I know the importance that like adults use strict typing and so I I refuse to grow up in that sense. It's it's a kind of rebellion. But I I just know that IT probably reduces the amount of errors even just for forget about performance improvements. IT probably reduces errors .

of when you do strict happy. So I mean, I think it's interesting. Look at that is give a hard time that there is this this cultural norm, this pressure is like there has be a right way to do things.

You know, grown up only do IT one way. Do that you should feel bad, right? Like some people feel like pythons, a guilty pleasure or something.

And that's like to get serious and going right. IT, right. Well, I mean, cool. I understand history and I understand kind of where this comes from, but I don't think he has big little pleasure. yeah.

So if you look at that, you say, why do you have to rewrite IT? You have to rewrite to deploy well, why do you want to deploy? Well, you care about performance.

You have about possibility. You want, you know a tiny thing on the server that has no dependencies or you know you have objectives you're trying to attend. So what if python can I achieve those objectives? So if you want types, well, maybe want types because you want to make sure you're passing on the right thing, sure.

You can add type if you don't care, you're protection some stuff. You're hacking some things out. You're like playing some RAM good of internet.

You should just work right? And he shouldn't be like pressured. He shall not feel bad about doing the right thing or the thing that feels good.

Now if you're in a team, right, you're working at some massive internet company. You have four hundred million lines of python code. They may have a house role that you use types yeah right.

Because that makes IT easier for different humans to talk each other, understand what's going on and box scale, right? And so there there are a lots of good reasons why you might want to use types, but that doesn't mean that everything should use them all the time, right? So what mojo does cool, allow people to use types. And use types.

You get nice things out of IT now, you get Better performance and things like this, right? But mojo is a full compatible super set of python, right? And so that means that has to work without types IT has to support all the dynamics things, has support all the packages that has support, uh, for comprehension, comprehensions and things like this, right? And so that starting point, I think, is really important.

And I think that, again, you can look at why I care so much about this. And there there's major from aspects that one of which is the world went through a very chAllenging migration from python two to python three, right? This this migration took many years and I was very painful for many teams, right? There's a lot of lot of things went on to that.

Um i'm not excEllent all the details I honestly don't want to be I don't want the world to have to go through that. Yes, right then you know people can ignore module if it's not, they're thing that's that's cool. But if they want to use mojo, I don't want them to have right all their .

code mean this OK the super set part is is just, I mean, so much billions stuff here that definitely is incredible. We talk about that first. Well, how's the typing implemented differently in in python verses mojo? So this had A A genius flexibility, do you said is definitely implemented?

Yeah so i'm i'm not a full expert and the whole export types and python. So i'll give you i'll give you that. I can give you my understanding.

Understand is basically, like many dynamic languages, the ecosystem went through a phase where people from different scripts to ring a large scale, huge, good basis in python, and at scale, kind of help some types. People want to be able to reason about faces, what what you expect, stranger and enter, like what these basic things. And so what the python community started doing is that started saying, okay, let's have tools on the side.

Check your tools, right? To go and like, enforce variance. Check for bugs. Try to identify, thanks. These are called sad analyst tools, generally.

And so these tools run over your code and try look for box. What happenings are. So many these things, so many different wear patterns and different approaches on specifying the types and different things going on.

The pilon community realized and recognize, 嘿嘿， hey, there's a thing here. And so what they start to do is they start to standardize the syntax for adding to pay them. Now one of the chAllenges that they had is they're coming from kind of fragment at world where there's lots of different tools.

They have different tradeoff, fs and pretax, and that has been different things. And so if you look at types and python according to the python spec, the types ignored, right? According the python spec, you can write pretty much anything in a tight position.

And uh, you can technically, you can write any expression. Now that's beautiful because you can extend that. You can do a cool things. You can right, build your own tools. You can build your house litter or something like that, right? There is also a problem because any existing python program may be using different tools and they have different reputations. And a few adopt somebody's package and your try around the tool you prefer in may throughout tons of weird ears mornings and problems just because it's incompatible with how these things were also because they are added late, they're not check by the past interpreters kind more a hint than this requirement also, uh, the c python implantation can use them for performance. And so it's really .

like the big one, right? So you can utilize the for the compilation for the Justin completion OK.

exactly. And this all comes back to the design principle of its it's the hints are kind of the definitions little bit markey is unclear exactly the interpretation bunch cases. And so because that you can't actually um even if you want to, it's really difficult use them to say like IT is going to be an if it's not as a problem, right? A lot of code break, we did that.

So so mojo, right? So you can still use those kind of type annotation. It's fine in mojo you declare a type and you use IT, then IT means IT is going to be that type and the compiler helps you check that and force that and it's safe um and it's not it's not a like best effort kind of a thing.

So if you try to show a string type thing into a from the compiler compiler time, nice. Okay, what kind of basic types are there?

Yeah so uh, module is a pretty hard core in terms of what IT tries to do in the language, which is the philosopher. There is that we um if you if you look at python, python is a beautiful language because it's so all right, all of the different things in python, like four loops and plus and like all these things can be access to these underground mbr methods.

Ah so you have to say, okay, if I make something that is super fast, I can go all the way down to the metal. Why do I need to have injuries built in the language? So what model does that says? Okay, well, we can have the notion of structure.

So if classes, and by them I have structure, classes are dynamic. Structure are static. We can performance.

We can write c plus plus kind of code with stocks if you want these things mix and work beautiful ly together um but what that means is you can go influence strength and ins and fruits and rain all I self in the language, right? And so that's really cool, because, you know, to to me, as A H I ideal, I idealizing compiled language type of person, right? What I want to do, I want to get magic out of the compiler input in the libraries because of somebody can know if we can build an injury, this beautiful and has an amazing A P I does all the things you d expect injured to do.

We don't like IT. Maybe we want a big in injure, maybe want to like sideways injure and like what what all the space of injuries are, then uh then you can do that and it's not a second class citizen. And so if you look at certain other languages like c plus plus one, I also love and use a lot um this hardware, the language, the complex is not.

And so I kind of weird that you have this S D D complex class. But you have int and complex tries to look like a natural numeric type things like this. But images and floating point have these like special promotion rules and other things like that. They're magic and they are hacked into the compiler. And because of that, you can actually make something that works .

like the built in types is something provided as a standard because um you know because it's A I first you know new murle types are so important here. So there's something 那个 nice standard implementation of into and flow。

yeah. So so we're still building on us to found. So we provide Andrews and float on the kind of stuff we also provide like a buffers and tensor and things like that.

We'd expect another context. Honestly, we need to keep designing and redesigning and working the community to build that out, make that Better. That's not our strengthening now give us six month or year and I think would be way Better. But um but the power of putting in the library means we can have teams of experts that aren't compiler engineers that can help us design and .

refine and drivers for so uh one of the exciting things we should mention here is that this is a this is new and fresh. This cake is unbaked. It's almost big, you can tell, delicious, but it's not fully to be consumed. Yes.

that's very fair. IT is very useful, but it's very useful if you're super low level programmer right now. And what we're doing were working away at the stack.

And so the way I would look at mojo today in may and twenty twenty three um is that is like a zero point one. So I think that you know you're for now to be way more interesting to a variety of people. But what we're doing, we decide to really see IT early so that people can get access to implant with them.

We can build with the community. We um have a big road map, fully publish, being transparent about this and a lot of people involved in the stuff. And so what we're doing is we're really optimizing for building. The same the right way and building the right way is kind of interesting working with the community because every wants IT yesterday. And so sometimes kind of know there's some dynamics there, but yeah, I think it's a the right thing.

So it's the discord also. So the dynamics is pretty interesting. Yes, sometimes the community probably can be very chaotic and introduce a lot of stress. Good to famously quit over the stress of the world. Operator, I mean.

maybe that they can be exactly.

And so like I could be very stressful as to develop but you can just attention upon attention is IT stressful to do a to work through the design of various features here given that the community so rely involved also um as i've been doing .

open development and community suffer decades now somehow this has happened to me so i've learned some tricks but the the thing that always gets me as I won't make people happy, right? And so is maybe not all people all happy all the time, but generally I I people to be you, right? So the chAllenge is that again, we're tapping some long some deep seated long tensions and pressures, both in the python balls and AI world and the hard world, things like.

And so people just want to move faster, right? And so again, our decision was let's release this early. Let's get people use to or access to and play with them.

And like, let's build in the open, which we could have, you know, had the the language monk sitting in the clutter up on the hill top, like be ring away trying to build something. But my experience, you get something way Better if you work with the community. And so yes, I can be frustrating and can be chAllenging for lots of people involved.

And I mean, if you you make our discord, we have over ten thousand people on the disco, one thousand people or something. Keep in mind release mode like two weeks ago yeah so um active so it's very cool. Um but what that means is that um you know ten one thousand people, I will want something different, right? And so what we've done as we've tried to say, okay, cool, here's a roman here here in the road map isn't completely arbitrary, is based on here's the logical order in which to build these features are at these keep doors.

And like that what we have done with fun really fast and like bug fixes. And so we actually have very few bugs, which is cool. I mean, actually for a project in the state. But then what we're doing, things were dropping in features very deliberately.

I mean, this is found the watch because you got the two jet communities of like hard work, like systems engineers, and then you have the machine learning python people that are like higher level yeah and it's just too like like army, like war e and so .

here's a talking. Okay, so here's again it's it's super funny .

for for something .

that only been out for two weeks, right? People are so impatient, right? But okay, cool.

That's fast for a year, like any years, time mode will be actually quite amazing themselves comes the problems be very good um people still have these problems, right? And so you you you look at this, you say in the way I look at this is to say, okay, well, we're solving big long standing problems to me. I again, working on many different problems.

I want to make sure we do IT right, right. There is like a responsibility feel, because if you messed up up, right, there is very few opportunities to do projects like this and have them really have impact on the world. If we do IT right, then maybe we can take those feeding armies .

and actually heal some of those wounds. Speech by something.

You look at this since, like, okay, well, how different early? Yeah, we all want beautiful things. We all want something is nice.

We all want to be able to work together. We all want to such be used, right? And if we can help kill that.

Now i'm not optimistic that all people use mojo on stop using class I K. That's not my goal. But but if we can hear some of that, I think that be prety cool .

yeah and and we start by putting the people who like races into the gug. No, uh, so so so there .

are proposals for acting braces to mojo. We just tell you just think we tell them, no.

okay, yeah.

Anyway, so there's .

a lot of amazing features on the road map in this writing. Be awesome. I can just ask if you think so. So uh, the the other performance improvement calls them immutability. So what's the what's this far and this let thing that we got going on, what's immutability?

Yeah so one of the things that is uh useful and not always required but useful is knowing whether something can change out or netho right? So and python, you have a pointer tune, ray, right? And so you pass that point, turn around the things you pass into function.

They may take that squad away in some other data structure. So you get your array back and you go to use IT. Now somebody else is like putting stuff in your area. yes.

How do you reason about that cause to be very complicated and these lots of bugs, right? And so one of the things that you know, again, this is not something module forces on you, but something ables, a thing called dimension. And what values metics do? They take collections like a race, my dictionary also sensors and strings and things like this.

There are much higher level and make them behave like proper values. And so that makes a look like a few. Pass these things around, you get a logical copy of all the data.

And so I pass you and ray. Sure, ray, you can go. Do you want to do, you know, can hurt my ray. Now that is an interesting and very powerful design principle, defines the way to of bugs. You have to be carefully implement in .

an efficient way as a performance hit that's significant.

uh, generally not if you implement the right way, but IT requires a lot of very low level getting the language right.

But I soon to be a huge performance hit because it's a really the benefit is really nice because .

you don't get into the trick is, is you you can't do copies, so you have to provide the behavior of copying without doing the .

copy yeah how do you do that is how do you do that?

It's not magic. It's just it's actually pretty cool. So first, before we talk about how that works, let's talk about how works in python, right? So in python need to find a person class or maybe person class is about you to find the database class.

And database class has a rare records like. And so the promise is that if you pass in a record or class instance into the database, it'll take a hold of that object and IT assumes that has IT. And if you're passing an object, then you have to know that that database is is gna. Take take IT. And therefore you should change .

IT after you put .

in the database. Know that, right? And so you roll out a version one of the database, you just kind of have to know that course lets users is own database, right? Yeah right.

Because you built, you understand this works, right? Some else joins the team. They don't know this, yes, right. And so now they are like get bugs. You're haven't maintained the database.

You shake your fist, you argue the ten time this happens, you're like, okay, we have to do something different, right? And what you do is you go change your python code and you change your database class to copy the record every time you added. And so what is the happening as you say? Okay, I will do called a defensive copy inside the database.

And then that way, if somebody passes something in, I will have my own copy of IT, and they can go do whatever, and they are not going to break my thing. Oh, this is usually the the two design patterns. If you look in pitch, for example, this is cloning a tensor there a specific thing and to call please, that's the state of your right.

So a different approach so using many languages so I work with that and swift um as you say, okay, well, let's provide value metics. And so we want to provide the view that you get a logical independent copy, but we want to that lazy. And so what we do is you say, okay, if you pass something into a function IT doesn't actually make a copy.

What actually does just increment the reference to? And if you pass around you sick in your date with, I can go on the day issue or not, and then you come back out of the stack, nobody copy anything. You come back out of back and then the color, let's go IT.

Well, then you've just handed IT off the database. You've transferred IT and there's no copies made. Now on the other hand, if you know your coal worker goes and hand your record and you pass IT and you stick in the database and then go fine, what happens is you get a copy lazily on demand. And so this does, this gives you copies only when you need them. And IT also so defines with the bugs, but also generally reduces the number of coffees and practice.

So but the importation details .

are tRicky here. I, yes, so something .

reference counting, but to make a performance, accost the number, different kinds of objects.

yeah. So you need a couple of things, so many. So this concept is existed in many different words.

And so vel research at all, right? The magic is getting the design right so that you can do this in eries. What we right? So there's a number of components to go into. This one is when you're passing around. So you we're talking about python and reference scanning in the expensive of doing that.

When you're passing values around, you don't want to do extra reference counting for no good reason and see you have to make sure that you're efficient to you transferred ownership instead of duplicating references and things like that, which is a very low level problem. You also have to adopt this and you have to build these data structures. And so if you say, um you know mojo has become part with python, of course, the fall list is a reference semantic list that works the way you d expect in python, but then you have to design a value semantic list. And so if you just have to implement that and then implement the logic with them. And so the the role of the language here is provide all the low level hooks that allow the author of the type to be able to get an express behavior without forcing into all cases or hard cutting this into language itself.

But there's a ownership. So Constance, transfer tracking.

who owns the thing? yes. So there's a whole system called owners. and. Related to work done on the rust community um also the swift community done a bunch of work and there's a bunch of different other languages that of all of c POS plus actually has copy constructors and destructive res and things like that. And so um and I mean this has everything so has moved constructors and has like this whole world of things. And so this is uh this is a body of work this kind of been developing for many, many years now. And so mojo takes some of the best ideas out of all these systems and remixes on nice way so that um you get the power of something like the rust programing language, but you don't have to deal that when you don't want to, which is a major thing in terms of teaching and learning and being able to use and scale systems.

Uh, how does that play with argument conventions? What are they? Are they important? How does the very romantic, how is the transfer ownership, uh, work with the arguments .

for their passenger? If you go deep into systems programing system, again, this is not something for everybody. But if you go deep in the asylum program and went what you encounter, you encounter these types that get weird. So you used to pass on, you think about everything I could just copied around. I can go change IT you to the song cool.

Um if you get into system programing and you get into these things like I have an atomic number or I have a new tex or I have a um uniquely owned database, handle things like this, right? So these types you can't necessary copy, sometimes you can't necessary even move them to different address. And so what motor allows you to do this that allows you to express, hey, I don't want to get a copy of this thing.

I want to actually just get a reference to IT. And by doing that, what you can say, you can say, okay, i'm defining something weird like atomic number or something like that has to be so so in atomic number is a an arian memory that multiple threads can access at a time without center, with without locks, right? And so um and so like the definition metallic numbers, more ple different things have to be poking.

Therefore they have to agree on where IT is. You can't just like move IT up for anyone because that kind of breaks what what IT means. And so that's that's an example of type.

You can even you can't copy. You can't move IT like when you could have to be where I was right now if you look at um many other examples like a day. So okay well, what happens? How do you copy a database and do you copy the whole database? That's not something you necessarily want to do.

Um there's a lot of types like that where you want to be able to say that they are uniquely owned. So there's always one of the thing and or if if I create a thing, I I don't cope. And so what module loves you do is love you to say, hi, I want to pass around the reference the same without copying.

And so has borough convention. So you can say you can use IT, but you don't get to change IT. You can pass IT by medical reference. And so you do that, then you can you get a reference to IT, but you can change IT. And so that manages all that kind of stuff.

That's just a really nice implementation. Make A C propose has yeah you know the different kinds of pointers yeah I smart, smart different different kinds of compositions of smart, explicit. Define this illogical but you're saying that's more like um the weirdo ase first .

is the common case well that depends I mean a Normal place to call all the people weird but the but you know if you talked to a Normal python a typical python programmer to really not thinking about this, right, this is a lower level of abstraction. And if you talked to people, certainly if you talk a rust programmer, again, there are not weird.

They are delightful like these are all good people right um those those looks the thing about all the time right? And so I look at this is there is a spectrum between very deep low level systems. I'm going to go poke the bits and care about how they're laid out memory all the way up to application and scripting other things like this. And so it's not that any boy's right wrong about how do we build one system of scales.

By the way, the the idea of an atomic number has been something that always brought me deep happiness. Because the fool p site, that the they did that threads can just modify stuff asy incurred sly. The whole idea of concurrent programing is a source of inner stress for me.

Well, so this is where you jump in to um you know again, you zoom out and get out of program compilers and you just look what the industry has done. My my mind is constantly blown by this, right? And you look at what you know, more law, more laws.

This idea that my computers, for a long time, single thread performs, got faster and faster and faster and faster for free. But then physics and other things intervened, and power consumption, like other things, start to matter. And so what other happenings? We went from single core computers to multi core than we want to accelerators. This trend towards specialization of hardware is only going to continue.

And so for years, us programing languages, nerds and compiler people have been saying, okay, well, how we tackle multiple for a while was like, model is the future we have to get on top of and most, of course, to the faul, what are we doing with the same and that is like there's chips with hundreds, of course. And what happened? right? yeah.

And so I am super inspired by the fact that on the face of this, you, those machine learning people, invented the sight of a tensor. Right was a tensor. A tensor is an like an horizontal and algebraic concept. It's like an ababde action around a gigantic paralyzed data.

And because of that and because of things like council and patriarch, we will say, okay, we will express the math of the system able to do our mac differentiations enable to you do like all these cool things um and and it's it's an abstracted representation because you have that abstract representation, you can not map onto these paralo machines without having to um control okay, put that back here, put that by there, put that by there. And this has enabled explosion in terms of AI compute accelerators like all the stuff. And so that's super.

super exact. What about the the deployment the execution cost multiple machines. So um you write that the module compute platform dynamically partisans models with billions of parameters and distributes their execution across multiple machines are enabling unparalleled efficiency. But the the use one parallel than that sentence anyway enabling on parallel efficiency scale the reliability for the largest workload. So how how do you do this um abstraction of a distributed the deployment .

of one of really a random walk through uh if you if you go back and replay the history of machine learning, right I mean the brief, the brief most recent history machine, very deep. I knew next one here in an AI podcasting, right? yeah.

So so if you look at just tensor patrol, which is pretty recent history in the next year, right? But tense flows all about graph s PyTorch, I think pretty unarguably ended up winning. And why did IT win? Mostly because the ability, right in the use ability of patric es, I think huge.

And I think again, that's a huge teston to the power of taking abstract, theoretical, technical cept and bring into the masses right now the chAllenge with what the tensor verse, the patriotic design points, was the kind of difficult to use for researchers, but was actually very good for deployment pid orchis, really good for researchers that kind of not super grate for deploy yet, right? And so I think that we, as an industry, have been struggling. And if you look at what deploying a machine learning mall today means is that you have researchers.

You are, I mean, wicked smart, of course, but the wicked smart model, architecture and data calculus. I like the working smart vary dems. They don't want to know anything about the harbour, the plane or floss things like this, right? And so what's happened is you people who train the model, they throw throw over the fans and they have people that tried to deployed the model.

Well, every time you have a team, a does x they threw over the fence. And team, why does some team? Team b is why, like you have a problem? Because, of course, they never worked the first time.

And so you throw over the fans, they figure out, okay, it's too slow, want to fit, doesn't use the right Operator, the tool crashes, what, whatever the promise and they have to throw back over the fence. And every time you throw a thing over a fence at three weeks of project manage and meetings and things like this, and so uh, what we see today is getting models. Production can take weeks or months like it's not typical.

And I talk a lots of people and you talk about like V, P, S software at some internet company trying to pull a model. And like, why do I need a team of forty five people? Like, it's so easy.

Train a model. Why can't I deploy IT? right? And if you dig into this, every layer is problematic. So if you look at the language piece, I mean, this is tip of the iceberg. It's a very exciting tip of the iceberg for folks, but you've got python on one side and see post post on the other side.

Python doesn't really deploy, I mean can therefore technically in some cases, but often a lot production teams will want to get things out of python because they get their performance and control and whatever else. So model can help with that. You look at serving, so you talk about gigantic models.

Well, a gigantic model won't fit on one machine, right? And so now you have this model is in python has to return the cost plus now IT also has be carved up so that half runs on aching have a runs shine or maybe IT runs on ten machines also. Now suddenly the complexity is exploding, right? And the reason for this is that if you if you look into transfer pyo systems, they weren't really designed for this world. They're designed for, you know, back in the day when we are starting and doing things where IT was a different, much simple world, like you want to run resonant fifty or some ancient model architectural like this, IT was just a IT was a completely .

different world that rain on one G, P, U.

exactly doing out one G P yeah and that right in the major breakthrough and um and the world is change right and so now the chAllenge is is a tender patrol systems. They were actually designed for L M. Like that was not a thing.

And so what where tensor actually has amazing power in terms of scale and deployment, things like that. And I think google is I mean, maybe not in match, but they're like incredible in terms that are capabilities and and gigged scale. Um many researchers using pyt, right.

And so I watch doesn't have the same capabilities. And so what modules can help with that now if he takes the back and say, like what is module doing? right? So module has like A A bitter enemy they were fighting against in the industry. And one of these things where everybody knows IT, but nobody is usually willing to talk about IT.

the bitter enemy.

the bitter thing that we have to destroy that we're all struggling with. And it's like all it's like fish gancy water .

is complexity. Yes, plea that's very close.

said. And so if you look at IT, yes, IT is on the harder side. excess. Go with the exit master complex over there.

Look at what's happening on the model inside massive amount of complexity, like things are changing the time people inventing, turns out the research is not done right, and so people won't be able to move as transforms are amazing. But there's a diversity even with the transformers. And what's the next transformer, right? And you look into serving also huge amounts of complex IT.

Turns out that all the club providers right have other very weird, but very cool hardware for networking, all this kind of stuff. And it's all very complicated. People are not using you look at classical serving right there.

There's this whole world of people who know how right I performed servers with zero ocock y networking like all all the fancy, uh, a sync I O and like all these fancy things in the in in the serving community, very little that has pervaded into the machine learning world, right? And why is that? Well, because again, these systems have been built up over many years.

They they haven't been rethought. There hasn't been the first principles approach to this. And so what module is doing? And we're saying, okay, we've built many of these things like so I worked on tensions and T P S.

And things like that. Other folks on the team like work on picture core, we've worked on on this one time. We ve worked on many these other systems. And so build systems like the apple accelerators and stuff like our team is quite amazing.

And so one of the things that roughly every mush is grumpy about is that when you're working on of these projects, you have the first or a goal, get the hard, hard to work, get the system to enable one more model, get this product out the door, enable the specific workload, make IT solve this problem for this park team, right? And nobody y's been given a chance to actually do that step back. And so an industry, we can take two steps forward.

We took like eighteen steps forward in terms of all this really cool technology across compiler's and systems and run times and heroines computer, like all this kind of stuff, like all this technology has been, you know, I wouldn't say, uh, beautiful ly designed, but it's been proven in different quadrants. No, you look at google, T, P S, massive, huge X A flops of compute strapped together into machines that researchers are programming python in a nobo. That's huge.

That's amazing. This is incredible. And so you look at the technology that goes into and the the algorithms are actually quite general.

And so lots of other hardware out and lots of other teams out there don't have the space cation or maybe the the years working on IT or the the budget or whatever the google does, right? And so they should be getting access same algorithms, but they just don't have that what my is doing or saying, cool. This is not researching more, right? We've built attuning in many systems.

We've built programing languages, right? And so I have you implement these plus, have implemented with have implemented many of these things. And so you know this is hard, but it's not research.

And you look at accelerators, what we know, there's a bunch of different weird kind of accelerators, but they actually cluster together, right? And you look at G P S, well, there's a couple of major vendors of G P S, and they maybe don't always get along, but their architectures are very similar. You look at C P S. C P S are still super important for the deployment ment side of things.

And you seeing new, new architectures coming out from all the club providers like this, and they're all super important to the world, right? But they don't have the thirty years of development that the in french people do, right? And so what modules can do is were saying, okay, all this complexity like it's not it's not bad complexity is actually innovation, right? And so it's innovation that's happening and it's for good reasons.

But I have symptoms for the poor software people, right? I mean, again, i'm generally soft person too. I love harder, but software people want to build applications and products and solutions that scale over many years.

They don't want to build a solution for one generation harbor with one vendors tools, right? Because of this, they need something that skills with them. You need something.

Were some cloud animal, right? Because, you know, the product is said I wanted to be have low latency, Better for personalization or whatever they decide, right? Products evolve. And so the chAllenge with the machine learning technology and the infrastructure we have today in industry, it's all these point solutions. And because they're all these point solutions, that means that as your part of ology like switch different technologies and what that does is that slows down progress.

So basically a lot of the things we've developed in those little uh, silos for machine learning tasks, you want to make that the first class citizen of a general purpose programing language, they can be compiled across all these kinds of hardware.

It's not really about the programme language. I mean, the programme language is a component of the mission. 嗯， the mission is are not literal, but our joking mission is to save the world from .

terrible A I software EXO.

So so you know, if you look at this mission, you need a syntax so that, so yeah, you need programme language, right? And and like, we wouldn't have to build the programming language. One existed, right? Self python was, are enough than cool.

We were just used IT, right? We're not just doing very large scale, expensive engineering projects for the sake of IT like it's problem, right? It's also about um uh accelerators. It's also about exotic numerals and beef lot sixteen and matrix implications and convolutions like this kind stuff within the stack, there are things like kernel fusion. It's a esoteric but really important thing that leads to much Better performance and much more general research hack ability together and .

that that's enabled by the asx that's enable by certain hardware. So like where's the dance between mr. Several questions here.

Like, how do you add a piece of hardware to the stack? Yeah, if you new piece, like if I have this genius invention of of a specialized accelerator, yes, how do I add that to the module framework? And also how, as modulate as a standard started, define the kind of hardware that should .

be developed. Yeah so let me take a step back and talk about that. So and so um if you go back to tense flow, one for one that this kind of time frame and these all involved in unity more complete.

So let's go back to the the glorious simple days, right? These things basically were C, P, U. And kua.

And so what you do, you say go to a dense lor and a denser or has a matrix molick, right? And so when you say that, you say, go to this big Operation, a matrix molick, and if it's on A G, P, U kick off could go, it's A C, P, U. Go to I can intel algar them or something like that with the intel M, K O, O.

Now that's really cool if you're either in video or right. But then more hardware comes in, right? And on one access, you have more hardwork coming in.

On the other hand, you have an explosion of innovation in A I. And so what happened with both tensile and patris that the explosion of innovation in A I has LED to is not just about make complication and computations. These things have now like two thousand different Operators.

And on other hand, you have I don't know how many pieces of harder there are. There is a lot, kay. It's not even hundred and is really thousands. And across all of edge and across like all all .

of the different things .

that are .

used that scale. exactly. I is not just like a handful of TPU alternative.

It's every, every phone, often with many different right chips inside of IT from different vendors, right? Like IT. It's AI is everywhere the thing, right?

Why are they all making their own chips? Like what? Why's everybody making their own thing?

Uh, will also be without a good thing special. So Chris is philosopher harbour, yeah, right? So my philosophe is that there isn't one right solution, right? And so I think that again, where the animal s law specialization happens again, if you if you're building if you're train GPT five, he wants some crazy supercomputer data center.

Thank you. If you're making a smart camera that runs on batteries, you want something looks very different. For building phone, you want something looks very different.

If you have something like a laptop, you want something that looks maybe similar, but a different scale, right? And so A I, and are touching over lives, robotics and like lots of different things. And so as you look into this, these have different power and the loves.

There's different trade off in terms of algorithms. There's new innovations and sparse data formats and things like that. And so uh, harder r innovation, I think is a really good thing, right? What i'm interested in is unlock king innovation.

There's also like anode in quantum and like although the the the really weird stuff, right? And so if somebody can come up with the chip that uses the analog computing and its hundred dex more proficient, think what that would mean in terms of the daily impact on the process we use that be huge. Now if you're building an animal computer, you may not be a compiler specialist, right? These are different skills sets, right? And so you can hire some compiled people of your right.

The company maybe, but IT turns out the these are really uh like exotic and new generation of compiler like this. This is a different thing, right? So if you if you take step back out and come back to what is the steps? Cw, so quite is that if you're intel, your video, you can you keep up with the industry and you chase.

And okay, there's one hundred now there's two thousand and three, twenty one hundred. And you have a huge team of people that are like trying keep up and two and optimize. And even when, uh, one of the bigs comes out with the new generation of their chip that to go back and rewrite all these things, right? So really, it's only powered by having hundreds of people.

They're all like practically trying to keep up. And what that does is that keeps out the little guys and sometimes they're so little as the big guys are also just not not in those dominant positions. And so um and so what has been happening.

And so a lot of you talk about the rise of new exotic, crazy cellars as people been trying to turn this from a let's go right, lots of special kernel problem into a compiler problem. And so we and I contributed to as well, an industry went to like, let's go make this compiler problem phase, let's call. And much of the industry still in phase, by the way.

So I wonder the phases over. And so the ideas say to say, look, okay, what a compiler does provides a much more general extensible uh hackable interface for dealing with the general case, right? And so um within machine learning algorithms, for example, people figured out that here if I do a max most location, I do a value right.

The classic activation function IT is wave faster to do one pass over the data and then do the value on the output when i'm writing up the data because really is just a maximum Operation maxie. And so it's amazing open ization take met more value squish together in one Operation. Now I have met more value.

Well, wait a second if I do that now, I just went from having no two Operas to three but now I figure out, okay, well, there's a lot of activation options. What about um uh likee rally? What about like like a million things out there, right? And so as I start fusing these, and now I get permutations of these algorithms, right? And so what the compiler people said, as I, hey, good, I will go mall and pair, and I will actually generate new.

And I think that this has been very, very useful for the industry. This is one of the things that powers google tps patroon oos. Like rolling out really cool compiler stuff.

Try in the other technology, things like this. And so the compiler people are kind of coming into their for and saying, like, awesome, this is a compiler problem, will compile IT. Here's a problem, not always a compiler person. I love compiler people, trust me, right? But not array canner should be a compiler person. Turns out the there are people that know animal computers really well, or they know some G, P, U, internal architecture thing really well, or they know some crazy Sparks amErica interesting algorithm is the custom research, but they're not computer people. And so one of the chAllenges with this new wave of technology, trying to everything took compiler again, is excluded ton of people. And so you look at what does mojo do, what is module stack do, brings program ability back into this world, like and enables, I would say, Normal people, but like a new, you know, a different kind of delightful nerd that that cares about numeric or cares about hardware, cares about things like this, be able to express that in the stack and extend the stack without having to actually go back .

and pilot itself to extend the stack on the, on the algorithm side, yeah. And then on the harder side, yeah.

So you can go back to like the simple example of end, right? And so both swept in mojo. And other things like this did, as we said, okay, pull magic out of the compiling, put in the library, right? So module is doing with the engine are providing, and like this, this very deep technology stack, right, which goes into other genes run times like how bunch really call really cool things um this this whole stack allows that stack to be extend and hacked and change by researchers and by harder readers and by people who no things that we don't know cause you much some smart people we don't have all the smart people turns out, right?

Ah, what are hero genius sometimes?

Yeah so so what is herding ious? right? So heroine just means many different kinds of things together.

And so the same simple example you might come up with is A C, P U, N, A G, P U. And so it's a simple hybrids computer. Say, I will run my data loading and preprocessing and other algorithms, ms, on the CPU. And then once I get IT into right shape, I have IT into the G, P U.

I do a lot of metro vacations and convolutions and things like this, and I get a back out, and I do some reductions and summaries, and I shovel across the wire, across the network to another machine, right? And so you've got now what are effectively two computers, A, C, P, U and A G P U, talking to each other, working together in a hergest system um but that was ten years ago. Look at a modern call phone, modern cell.

You've got CPU and they're not just C P U. There's like big dot, little C P U, and there's most people, different kinds of C P S are. And working together, most core you to get G P S, you've got no network excEllence.

You've got dedicated harbor blocks for media video decode and a decode and like. And so you've got this massively complicated system. And this isn't just sulfone every laptop these days of doing the same thing, and all these blocks can run at the same time and need be choregraphic.

And so again, one of the good things about machine learning is it's moving things to like date flow, gress and higher level of abstractions and tensor and these things that IT doesn't specify. Here's how to do the algorithm IT gives the system a lot more flexibility in terms of how to translator matter, compile IT onto the system you have. And so what you need, you know, the bottom is part of the layer.

There is away for all these devices. Talk to each other. And so this is one thing that you are very passion about.

But all these, all these machines, systems are effectively parallel computers running at the same time, sending messages to each other. And so they're all fully a synchro. Well, this is actually a small version of the same problem.

You have a date to center, right the data center. You now have multiple different machines, sometimes very specialist, sometimes with G P S or T P S, and one now and sometimes with disks. And so you get a much larger scale header gies computer. And so what ends up happening is you've have this like multi lare abstraction of hierarchical parallel, hierarchical ec chance communication. And making that, again, the end my enemy is complexity, by getting that away from being different specialized systems at every different part of the stack and having more consistency and uniformity, I think we can help lift the world and make IT much simple and actually get used.

But how do you leverage like the strength of the difference system? So looking inside the smart one, yeah, like what I know, five, six computers and centrally inside smart uh how you uh without trying to minimize the explicit uh making A A which .

computers for Operation. So you're doing you're looking at two, two factors. You're looking the factor of sending data from one thing to another, right? That takes time to get IT from that side of the chip to that of the chip, things like this.

And they're looking at what is the time IT takes to do an Operation on a particular block. So take C P S. C P S are fully general.

They can do anything, right? But then you have a etic that's really good a matter most locations and say, so okay, well, if my workload all max complices, I start up, I send the data over the neal. That thing that goes to the complications when is done is something back.

The result, I was good. The simple thing is just saying, do you do trc Operations over there? But then you realized you get a little bit more complate because placation A G P U, you can do IT.

Um no metics you can do on C P U. And they'll have different trails and cost, and it's not just makes molick. And so what you actually look at is you look at I have generally a graph of computer. I want to do a partitioning.

I want to look at the communication, the bisection ban with them like the overhead in the sending of all these different things, and and build a model for this, and then decide, okay, it's an opposition zo problem. Where do I? I want to place this compute.

So the old school, the real computer science problem of scheduling. And then how does A. Presumably it's possible to somehow magically include other june to this?

absolutely. So I mean, in my opinion, this is opinion. This is not, uh, not a really degree with this. But in my opinion, the world benefits from simple and predictable systems at the bottom that you can control. But then once you have a predictable execution later, you can build lots of different policies on top of IT. And so one policy can be that the human programmer says, do that here, do that here, do that here, do that here.

And like fully manual controls everything and the system should just to do IT right, then you quickly get in the mode of like I don't want to have to tell you to do IT yeah and so the next logical step that people typically take as they write some terrible heroic if it's a mage location, do over there or if it's floating point, do on the G P. Of its injured due on the CPU, like something like that, right? And and then you you then get into this mode of, I like people care more and more more and you say, okay well it's actually um like make here Better.

Let's get to other to s actually do uh search of the space to decide well what is actually Better right? Well then you came to this problem. You realized this is not a small space.

This is a many dimension, hyper dimensional space that you cannot exhaustively search. So do you know any algorithms are good searching? Very complicated is for.

don't tell me, you're gonna turn this into a machine learning problem.

So then you turn into a machine learning problem, and then you have a space of tuning. You include into the stack.

into the, into the module.

Yeah, yeah.

Where is the sit? Where is the love? Is a separate thing as a part of the compilation.

So you start from simple and predictable models. And so you can have full control and you can have course grain, not like systems, and have to do this. But if you really care about getting the best, the last outside of a problem, then you can use additional tools.

And there are cool thing as you don't want to do this. Every time you run a model, you want to get the right answer on them, cash IT. And once you do that, you can get, you can say, okay, cool.

I can get up doing very quickly. I can get good execution out of my system. I can decide if something important. And if it's important, I can go through a bunch machines at IT and do a big expensive search over the space using whatever technique I feel x product to the problem.

And then when I get through the answer, cool, I can just start using IT and so you can get out of this um this trade off between, okay, I onna like spend forever doing a thing, or do I get over quickly? And is a quality result like these are actually not in contention with each other? If the system design scale.

you started in the little bit of a world wind overview of how you get thirty five thousand x uh speed up or more of a python um germy Howard did a really great presentation about sort of the basic we will get the code, here's he get the speed up like he said, that something we can probably developers can do for their own code to see how you can get this jack giant. But he may be, speak to the machine learning task in general.

How do how do you make some of this code fast specific? Like what would you say the main bottom like, uh, for machine learning that are we talking about, matt, more matrix modification. How do you make their first?

So I mean, if you just look at the python problem, right, you can say, how do I make python faster? There have been a lot of people that have been working handy. Okay, I don't make python to investor ten investor something like there's of projects in that then, right? Mojo started from the what can the harbor do? like? What is limited physics? yeah.

What the speed of flight? What like? How fast, fast and go. And then how do I express IT? Yeah right.

And so wasn't wasn't anchor relatively on make python imo, but faster, cool. I know what the hardware can do. Lets and lock that right now.

When did you say how how good that is to be in the meeting? And as supposed to try, see how do we get the improvement? Like what can the physics do?

I mean, maybe i'm a special kind of but you look at that, what is the limit of physics? How fast can these things go right? When you are looking that typically IT ends up being a memory problem, right? So today, uh, particularly with these specialized accelerators, the problem is that you can do a lot of math within them, but you get bottle that sending data back and forth, the memory, whether be local memory or distant memory or disco whenever IT is and and that the bottle particular is the train sizes get large as you start doing tons of inferences, a little place like that becomes a huge bottle neck for people.

So again, what happened is we went surface of many years where people took the special case in hand tuned IT twick didn't track down, and they knew exactly how the hard to work, and they knew the model they made. They made IT fast, didn't journalist. And so you can make, you know, recent fifty or some or or something inception v one like you can do that right? Because the models are small, they fit in your head, right? But as the models get bigger, more complicated, as the machines get more complicated, stops working, right? And so this is where things like corn fusion come in.

So what is turn off fusion? This is the idea of saying, let's avoid going to memory and let's do that by building a new hybrid kernel and a numerical algorithm that actually keeps things in the accelerator, said, having right all way out memory. What's happened with with these exciters? Now you get multiple levels of memory, like in A G, P U, for example, you'll have global memory and local memory like all these things.

Um if you zoom way into how harder works, the register file is actually a memory. So the registered are like an elza cash. And so a lot of taking advantage of the hardware ends being full, utilizing the power in all of this capability.

And this has a number problems, right? One of which is, again, the complexity disaster, right? There's too much hardwork, even if you just say, let's look at the chips from one line of venture like apple or intellect, whatever is.

Each version of the chip comes out with new features and they change things. So that takes more time or less time to do different things. And you can't rewrite all the software whenever new trip comes, right? And this is why you need a much more scalable approach.

This is what moto and what the module stack provides, provides this infrastructure in the system for factory, all this complexity, and then allowing people to express algorithms. You talk about attuning, for example, express algorithms in a more portable way so that when a new trip comes out, if you don't have reread ital. So to me, I know I kind of joke like, what is a compiler? What many ways to explain that? You convert thing and thing being.

You convert source code machine. You you can talk about many, many things the compilers do, but to me it's about a bag tricks. It's about a system and a framework you can hang. Complexity is a system that can then generalize.

And I can work on problems that are bigger than fit in one human head, right? And so what that means? What a good stack and what the module stack provides the body to walk up to IT with a new problem and generally work quite well.

And that's something with a lot of machine learning infrastructure and tools and technologies don't have a typical state. They are today. As you walk up, you're delaying. If you walk up with the new model, you try to push IT through the converter, the converter crashes. That's crazy.

The state of ml tooling today is not anything that A C programmer would ever accept, right? And it's always been this kind of flaky inside of tooling has never been in great well and it's spent uh, never work together. And because it's not designed together, has built by different teams, is built by different different vendors, is built by different systems, has built by a different internet companies and transsolar ve their their problems, right? And so that means that we get this fragment terrible, massive complexity.

So I mean, the specifics of amy journey showed this. There's the vectra zed function, which he goes, uh, built in to the inter mojo variations .

as he showed us.

built into the library, into the library, in the library factorization paralyzes, which vectorially es more low level, paralyzes high level. There's the thailand thing, which is how he demonstrated the auto tune. I think so.

So think, think about this. And like levels, hierarchical levels of abstraction, right? So at the very fusan, all the way into a computer problem, you have one floating point number, right?

So then you say, okay, I won't be. I can do things one of the time in an entrepreneurship. Ty slow, right?

So I can get to doing one of time in a compiler. I can see. Then I can get to doing four, eight or sixteen of the time with factors that called factors ization.

Then you can say he A A bunch of different, you know what? What a multicolored puter is is, is basically a bunch of computers, right? So there are all independent computers that they can talk each other and they share memory.

And so now what parallel does he says, okay, run multiple instances on different computers. And now they cannot work together. And from right, so what are you doing? interesting. Keep going out to the next level out.

And and as you do that, how do I take advantage this so telling is a memory optimization, right? And says, okay, let's make sure that we're keeping the data close to the computer part of the problem instead of sending IT all back. And for three memory, every every time I look a block and .

size of the block sizes as well as I get to the attune make you promise .

yeah also all of these the details matter so much to get good performance um this is another funny thing about a machine learning and high performance computing that is very different than see compilers.

We all grow up, grow up with where you if you get a new version of G C C or new version of clank or something like that, you know, maybe something will go one percent faster, right? And so compilers really, really, really hard to get half a percent out of your c code, something like that. But when you're talking about an accelerator or an air application or you're talking about these kinds of algorithms.

And these are things people is right for train, for example. Right if you get IT wrong, it's not five percent or one percent that can be two x or ten x right? If you think about IT, um you really want to make use of the full memory.

You have the cash, for example. You too much space IT doesn't fit the cash. Now you're going to be thresh ing all the way back out to main memory. And this can be two x ten x major performance differences. And so this is where getting these magic numbers and these sounds right is really actually quite important.

You mentioned that more just super set of python. Can you run python code as a face module code?

Yes, yes. So so in this has two sides of IT, so modules not done out. So i'll give you this claim. But already we see people that take small pieces of python code move IT over. They don't change IT.

And you can get twelve of x edms like someone just tweet about that yesterday, which is very cool, right? And again, interprets compilers, right? So without changing, I could without also is not this is not jack compiling or do something, anything fancy, just basic stuff.

Ministration over now, mojo will continue to grow out. And as IT grow out, IT will have more, more, more features and are our stars to be a false super side of python. And so you can bring over basically arbitrary python code and have IT just work and IT may not always be twelve that faster, but um but IT should be at least as fast and way faster in many cases. Um now i'll take time to do that in python is a complicated language, not just the obvious things, but there's also not obvious things that are complicated like we have people will talk to c python packages to talk to the C A P I and there's A S A pieces.

So you have to to make explicit the obvious, may not be so good。 So you think about IT. So you know, to run python code that is after run all the python packages libraries yeah so that means what what's the relationship between a gencer path on the the interpreter that presumably would be task with getting those packages to work here?

So in the forest of time, mojo will solve for all the problems and move python pages over and run dom in moto without .

the sea pya without sea python.

someday. Yeah, right? Not today. someday. And that will be a beautiful day because then you will get whole bunch of advantages and you'll get massive speeds.

ups and things like this. So you can do that one of the time, right?

You can move package exactly, but we're not willing to wait for that. Python is too important. The ecosystem is too broad. A we want to both be able to build mojo out.

We also want to do IT the right way without time, like without intense time pressure were obviously moving that but um and so what we do is we say, okay, well, let's make us you can important arbitrary existing package arbitrary including like you write your own on your local desk, whatever is not is not like a stand back, like an arbitrary package, and import that using sea python, because c python are IT runs of ages, right? And so what we do is we built an integration layer where we can actually use sea python. Again, i'm practical.

And to actually just load and use all the existing packages as they are. The downside of that is you don't get the benefits of motor for those packages, right? And so the run as fast as they do in the traditional c python way.

But what that does is that gives you an incremental migration path. And if you say, hey, cool. Well, here's a you know the python ecosystem that I want all the to just work.

But there are certain things are really important and I am doing weather forecasting or something. 我要 i want to be a little all the data want to be a work with that。 Then I have my own crazy algorithm inside of IT. Well, Normally I D write them, see POS plus second mojo, have one system that skills, well, that's way easier to work with.

Is IT hard to do that to to have that layer that's running sea python because is there some communication back and force? Yes.

it's complicated. Mean is what we do. So I mean we make IT lucky easy, but um it's IT is complicated. But what what we do is we use the sea python existing interpreter because running its own bike codes and not how provides for compatibility. And then IT gives us sea python objects and we use those objects as is.

And so that way we're fully compatible with all the c python objects and all the um you know it's not just the python part is also the sea packages. The sea library is underneath them because they're often hybrid. And so we can fully run and we're fully compatible with all that. And the way we do that is that we have to play by the rules. And so we we keep objects in that presentation when they're coming from that world.

What's the representation is being .

used in memory would have know a lot about how the c python interpreter works. IT has, for example, reference counting, but also the different rules on how to pass pointers around and things like this super low level fitly. And it's not like python. It's like how the interpreter works, okay? And so that gets all exposed out.

And then you have to define rappers around the low level, right? And so what this means that you have to know, not only see which is a different role from python oba, 所以 not only python， but the rappers, but the interpret and the rappers and the influences, details and the conventions. And I just is really complicated mess.

And when you do that now, suddenly you have a debugger, the debug python, they can step in the c code. You get this two world problem, right? And so by pulling this all in to moto, what you get, if you get one world, you get people to say, could I have untypical very dynamic, beautiful, simple code? okay? I care about performance for whatever reason, right? There's lots of reasons you can you you might care.

And so then you at types, you can paralyze and you can veterans and you can use these techniques, which are general techniques to solve problem. And then you can do that by staying in the system. And if you uh, you have that, that one python tax is really important to you, you can move to the you get massive performance benefits on that.

Another other advantages, you know, if you like exact types, it's nice, but their enforced some people like that, right, rather than being hints. So there's other advances too. And then um and then you can do that incremental as you go.

So one different perspective on this will be why mojo, instead of making sea python faster, redesigning c python yeah I mean.

you can argue mojo is redesigning c python should, but uh but why not make sea python faster and Better? And other things like there's lots people were him. So actually there's a team of microsoft that is really improving, I think.

See, python three thousand eleven came out october, something like that. That was you know fifteen percent faster, twenty percent faster across the board, which is pretty huge given how mature python is and things like this. And so um that's awesome.

I love IT. Doesn't run G P U， doesn't do A I F like he doesn't do vectors, doesn't do things um twenty percent good thirty five thousand times is Better, right? So like they are definitely i'm a huge fan that work, by the way, and IT composes well with what we're doing.

And so it's not like we're fighting or anything like that. It's actually just general goodness for the world, but it's just a different path, right? And again, we're not working forward from making python a little bit Better or working backwards from most .

limited physics. What's the process of porting python code to modes there? What's involved in that in the process is there are tooling for that not yet.

So um we're missing some basic features right now and so are continuing to drop out new features like on a weekly basis. Um you know at the fullness of time, give us year and half, maybe two years.

Is that an automated process?

So already there will be very automated.

yes. Is automate automate like it's possible to automate. In the general case, the bill of the conversion is possible.

This is why, I mean, among other reasons why we use tabs. yes. right.

So first, while by being a super cent, yeah you it's like server s six plus. Can you move c code to c POS plus? yes.

Yeah right. And you move you you can move c code to C S plus. And uh then you can adopt classes, you can adapt temperature, you can adopt other references or whatever C P, S lus features you want. After you move, see sea code to sea pass plus like you can use temples and sea right. And so if you leave to see if fine, if you can use the cool features, but is still works right, and see people could work together.

So that's the ology right now um here right you you you there is not a python as bad and modules good right? Mother gives you superpowers, right? And so if you want to stay with python, that's cool. Uh but the tooling should be actually very beautiful and simple because we're doing the hard work of .

defining a super cent. You're right. So the several things to say that but also the conversion tooling should probably give you hints to like how you can improve the code and .

then exactly when you're in the new world, then you can do to all kinds of cool tools to say, OK should you adopt this feature or like, and we haven't those tools, yep, I fully expect those tools to exist. And you like modern ize code, how you want to look at IT.

So I mean, one of the things that I think is really interesting about mojo is that there have done a lot of projects to improve python over the years um everything from you know getting job a python around the job of virtual machine, uh, pipi, which is the jit compiler. There's tons of these projects out there. They're been working on improving python in various they found the one two camps.

So pipi is a great example of a camp that is trying to be comparable with python. Even there not really doesn't work with all to see packages and stuff like that. But um but they are trying to be compared with python.

There's also another category of these sgs where they're saying, well, python is too complicated and i'm gna cheat on the edges and you know like ines in python can be an arbitrary sized and injure like you care about fitting in a going fast and a register in a computer that's really annoying, right? And so you can, you can to choose two pass on that, right? You can say, well, people don't really use big injuries that often.

Therefore, i'm going to just not do IT and that will be fine. You not not a python super set or you can do the hard thing and say, okay, this is python. You can be a super set of python without being super set python. And that's a really hard technical problem.

But it's, in my opinion, worth IT, right? And it's worth that because it's not about anyone packages about this ecosystem, about what python means for the world and also means we don't want to repeat the python two to python three transition like we want. We want people to be able to adopt this stuff quickly. And so by doing that work, we can help the people.

Yeah, the chinese really interesting in technical, physical, hc chAllenges, really making a language, a supermarket of another language, breaking my brain a little .

bit but well a pension on the corners. So um again, i'm very happy with python. And so joking all joking side, I think that the annotation thing is not the actual important part of the problem, right? But the the fact that python has amazing dynamic meta programing features and they translate the beautiful static programme features I think is found, I think that's huge, right? And so python, i've talked with you about this.

It's like IT was not designed to do what we're doing. That was not the reason they built IT this way, but because they really cared, and they are very thoughtful how they design the language. IT scales very elegantly in the space.

But if you look at other languages, for example, cnc pass plus, right, if you're building a super set, you get stuck with the design decisions of the subset, right? And so you know seat plus plus way more complicated because of sea and legacy, then IT would event if they would have theoretically designed a from scratched thing. And there's lots of people right now they're trying to make C S plus Better in recent tax.

C POS. Plus can be great or just change the tax. But if you do that now simply loves your practice, you don't.

So what what are the if you could just linger on what of them? Biggest chAllenges of keeping a supermarket status, what do the things just strugling with?

Is IT all ball down to having a big in order? No, I mean usually the it's a long hill, weird thing. So let me give you a worst story.

okay. So war story in the space is, um you go away back in time. Project I worked on is called clang, clang.

What is C C S plus parsons right? And when I start working and playing has been my two thousand six or something I two thousand and seven two thousand six when first start work on right um funny how time fly ah yeah the um I start that project and i'm like, okay well I want to build A C para c propose for L V M. It's gonna be the world.

G C C is yuki you know this is mean earlier times as yuki is on principle that has always word features like all these bugs like, is jackie. So gonna build a standard compliant cn, can be beautiful, would be amazing, well engineered, all the ool things and engineer wants to do. And I start inflated building out, building on, building out.

And then I got to include standard diode age and all of the letters in the world geologic stuff. This, so again, come back away from theory, back to reality, right? I had was at a folk on the road, I could build an amazingly beautiful academic thing that nobody would ever use.

Or I could say, well, it's yuki in vous ways. All these design mistakes, accident history, the legacy that point, G C C was like over twenty years old, by the way, right now l vms over twenty years old. Yeah time matches up you right? So um you you say, okay, well, what what what is easier, right? I mean, as an engineer, it's it's actually much easier for me to go implement long tail compatibility word features even if they are distasteful and just do the hard work and like figured out of our engineer, understand what IT is right bunch test cases like try to understand behavior. It's way easier to do all that work is an engineer than to go talk to all c programmer and get argue with them and try to get .

them to rewrite their code yeah right and cause that breaks a lot .

more things yeah and and you have realities like nobody actually understands how the code works because was written by the person who quit ten years ago, right? And so this this software r kind of frustrating that way, but it's that's how the world works yeah.

unfortunately, can never be this perfect, beautiful thing.

Well, there are occasions in which you get to build like, you know, you invent new data structure or something like that. There is a beautiful algorithm that like makes you super happy. And I I love that moment.

But so when you're working with people, yeah you're working with code and dusty, that code bases and things like this, right? It's not about what's directly beautiful, about what's practical. It's real, what people actually use. And I don't meet a lot of people that say I want to rewrite all my code just by the way.

that could be interesting possibilities. Probably talk about where A, I can help provide IT some code that maybe further out feature was a really interesting one. How that could create more be A A tool in the battle against the monster of complexity, the imation.

You mentioned good to the the benevent lent dictator for life of python. What do you think about moger? So you talked too much about .

IT um I have talked with him about IT found a very interesting um we actually talked with quito before we launched. So one public um I have a ton spect for great to for a bunch of different reasons. You talk about the Operator and I like gres.

Pretty amazing in terms of string, such a huge universe community and and like driving forward. And I think python is what IT is thanks to him, right? And so to me, he was really important starting to work on mojo to get his feedback and get his input and get his eyes on this right now.

Um a lot of what we do was is wasn't as I think and think about is have we not fragment community yeah we don't want to py on two to python three things like that was that was really painful for everybody involved. And so we spent quite a bit time talking about that and some of the tricks I learned from swift, for example. So in the migration from shift, managed to like not just convert objective sea into a slightly pretty objective sea, which we did, we then converted not entirely, but almost an entire community to completely different language, right? And so there's a bunch of trick sea learned along the way they're directly relevant what we do.

And so this is where, for example, the you ever to see python will bring up the new thing like that. That approach is, I think, proven and then comes from experience. And so great to was very interested in like, okay, cool.

I think the python is really his legacy, his baby. I have time to respect for that. Incidently, I C mojo is a member of the python family.

Not try to take path on away from grade and from the python community. Um uh and so uh to me it's really important that were a good member that community. So I think again, you'd have to ask you to this but I think he is a interested in this notion of like cool. I think it's been up for being slow.

Maybe there's a path out of that, right? And that, you know the future is python, right? I mean, look look at the the the far outside case on this, right? And i'm not saying this is greet as perspective, but you know there's this path of saying like, okay, well, suddenly python, I go all the places is never been able to go before, right? That means the past I can go even further and can have even more impact on the world.

Some module could be seen as piton for all.

I would not say that I I would drive a lot of people really .

crazy because of the P T, S, D of the .

three point or two .

i'm willing .

to annoy people about my.

But the point is the step of being a super set and a lying. All these capabilities, I think, is the evolution of a language. IT feels like an evolution of the language, so he is interested by the idea that you are playing with, but also concerned about the figure table.

So how what are the ideas you've learned? What are you thinking about how we avoid fragmenting in the community? Or the the the python sters? And I don't know what to call the mojo people. I an jian. I like IT exist happily and share code, and basically just have these big code bases that are using a sea p, python and more and more movie or moger.

So these lessons I learned from swift, and here we face very similar problems, right? And shift, you have objective c super dynamic. Uh, there are very different in tax, right? But you're taking people who have large scale code basis.

I mean, apples got the biggest, largest scale code basis. And so no, none of the companies is not the IOS developers, not the other developers, want to write everything once ince, he want to be able to adopt things piece of the time. And so I think that I found the work very well.

And the swift community was saying, okay, cool. And this is when so which is very Young, and you say, okay, you have a million line of code object, C, P, don't rewrite IT all. But when you influence in new feature to influence that new class using swift, right? And so now this turns out is a very wonderful thing for an APP developer, but is a huge chAllenge for the compiler team and the system.

People are implemented. And this comes back to what is this trade off between doing the hard thing that the able scale versus doing the thereafter ally peer r, an ideal thing, right? And so swifts had adopted and built a lot of different machinery to deeply integrate with the objective runtime.

And we're doing something with python right now. What what happened in the case of swift is swift as a language got more, more, more mature over time, right? And incidentally, modules are much simpler language than swift in many ways.

And so I think that module will develop wave faster than swift for a variety reasons. Um but as a language gets more mature in parallel with that, you have new people starting new projects when the language is mature and they are a new project, that's when they say, okay, cool. I'm not dealing with a many lines of code.

I'll just start and use the new for my whole stack. Now the problem is, again, you come back to our communities and where people that work together, you build new subset feature and a thing in swift, or you build anything in mojo, then you want to be end up being used on the other side, right? And so then you need to work on immigration back the other way.

And it's not just motor talking. Python is also right. And so what I would love to see, I don't want to see this next month, right? But when I want to see over the course of time is I would love to see people that are building these packages like you know no pie or uh you know TensorFlow or what you these packages are half python, half sea plus plus.

And if you say, okay, cool, I want to get out of the python c plus plus world into a unified role and so I can move to module, but I can't give up on my python clients. There's like these levers used by everybody. They're not all going to switch every you know all once and maybe never, right? Also, the way we should do that as we should then python interfaces to the module types.

And that's what we didn't swift and work great. I M was a huge influences chAllenge for the compiler people, right? But um there's only a dozen of those compiler people and there are millions of users.

And so it's a very expensive capital intensive like skill set intensive problem. But once you solve that problem, that really helps adoption and really helps the community progressively adobe technology. And so I think that this approach al work quite well with the python on the motor world.

So for a package ported to module and then create a python interface. So how do you just link on these packages? no.

pyto. Ch TensorFlow, yeah, how do they play nicely together? So is mojo supposed to be? Let's stuck bottle machine learning ones. Is mojo kind of vision to replace py ocean? TensorFlow, uh, to incorporate what's what's the relationship and is right?

So ms dance, so take a step back so I wear many heads so you're you, you're anglin and on the mother said, yes, modules, the program of language and so IT can help solve the C.

C, P plus python view we should .

be talking for. So so I love IT. It's it's a big deal.

The other side of this is the fire movies and service of solving some bigger problems, right? And so the bike AI problems are, again, this fragmentation, the harder nightmare. This is the explosion of new potential.

But that's not getting felt by the industry, right? And so when you look at how does the module engine help cancel them by ord, right? It's not replacing them, right? In fact, when I talk to the people again, you don't like to rewrite other code. You have people are using a bunch of PyTorch, a bunch of TensorFlow.

They have models that they have been building over the course of many years, right? And when I talk to them, there are a few exceptions, but generally they don't want to write other code, right? And so what we're doing, we're saying, okay, well, you don't have to rewrite all your code.

What happens is the modular engine goes in there and goes underneath. Tensor patroons is fully compatible and just provides Better performance, Better prettily, Better tooling is a Better experience that helps lift transfer m and pitch to make them even Better. I love python.

I love tension flow. I love pyrite. This is about making the world Better because we need to go further.

But if I have a process, the trains the model and have a process that performs influence on that model and have the model itself, what should I do with that in the long arc of history? Um in terms of if I use python to train IT, should I rewrite stuff and mojo with that? If I care about performance oh .

so I mean again, IT IT depends if if you care about performance, then writing and moles can be way Better than writing in python. But if you look at um if you look at for example, so you look at open eye removed and you look at many the other folks are working on may these many these elms and other like innovative machinery models.

On the one hand, they're innovating in the data collection, in the model, billions of premiers in the model architecture and the r yeah H F, like all the, all the cool things of people are talking about. But on the other hand, staying long time, writing cute girls. And so so wait a second.

How much faster could all the progress go if they were not having a handout? All these cuts, right? And so there are a few technologies out there. People have been work in problem for a while and um and the try self substance to the problem again, kind of ragging the space and what module provides for these kinds of companies is the ability to say cool. I can have a unifying theory right again, the the Better together, the unifying theory, the two real problem, the three world problem, the general problem like this is the thing that are slowing people down. And so as we help solve this problem, I think that would be very helpful for making the whole psychotic faster.

So obviously we ve talked about the transition from objective sea to swift design this uh programing language and you've also talked quite a bit about the use of switch from machine learning uh context. Why have you decided to move away from uh, maybe an intense focus on swift for the machine learning context versus of designing a new programing language that happens to be a super .

set is an irrational set of life choices I make.

Did you go the desert and did you meditate on IT? okay. Right now, I was boit was bold and needed. And I think I mean, it's just bold. And sometimes to take those leaves is difficult .

if to take yes. And I think there's a couple of different things. So um actually I left apple back in twenty seventeen, like january twenty seventeen.

So a number of years that I love apple. And the reason I left apple to do air. okay. So and again, I won't come on appi, but the uh uh at the time right I want to get into and understand and understand the technology, understand the applications works. I'm going to dive deep into apply A I and then the technology needs that, right?

I found myself a google, and those I can. T, P, U were waking up.

exactly. I google, uh, jeff teen, whose rockstars, you know, right? And the and twice seventeen tense flow, like really taking off and doing credible things. And I was attracted to google to help them with the T. P S.

right? T P S are an innovative, hard work, salary or platform I have now, I mean, I think proven massive scale, like done incredible things, right? And so one of the things that the sled into is a bunch of different projects, which all skip over, right? One of which was the swifter tensor project, right? And so that project was a research project.

So the idea of that is, say, okay, well, let's look at innovative new programmer models where we can get a fast programme language. We can get automated differentiation into language, push the boundaries of these signs in a research setting right now. That project I think lasted to three years.

There's some really cool outcomes of that. So one one of the things that's really interesting as um I published A A talk at an elegant conference in twenty eighteen and seems like so long go a about graph program of abstraction, which is basically the thing with in pitch ch two. And so pitch ch two all the time are real thing is all about the graph program abstraction thing from python bike.

And so a lot of the research that was done um and pursuing and going out through the industry and influencing things, and I think it's super exciting and awesome to see that. But the super tensile project itself did not work out super or. And so there's a couple of different problems with that, one of which is that you may have noticed. So if is not python, there's a few people the right python code, yes. And so IT turns out that all of them I was pretty happy with .

my thon is actually a problem. Other programing languages just have as well that they not python. 我 probably may be briefly talk about Julia was a very interesting beautiful programing language.

but it's not python. And and so like if you're saying i'm going to solve a machine learning problem where all the programmer or python programmer yeah and say the first thing, you reduce switch to different language. Well, your new thing, maybe good or Better, whatever, but if it's a new thing, the adoption arrier is massive.

So possible, so possible. Yeah, absolutely. The world changes and evolves. And there's definitely for new, new good ideas. IT just makes you so much hard, right? And so lesson learn. So if if is not python, and people are not always in search of like learning anything for the second, learning anything. And if you want to be compatible with all the world's, could turns out me the world word is right.

Second thing is that um you know lesson learn is that uh swift as a very fast efficiently, which is kind of like mojo, but different different take on IT still um really workout with egg er mode and so eager mode is something that patrols does and IT proved out really well and IT enables really expressive and dynamic and easy debug programme. Um tensor of the time was not set up for that. So that was not .

the timing is also important in this world.

Yeah yeah thanks for a good thing. Has many, many strength, but a you can say sf for tensor is a good idea except the swift and accept the tense flow part because it's not .

python tens of floor because .

it's not not at the time.

Yeah one point.

And so one of so one of the things about that is in the context of IT being a research project, i'm very happy with the fact that we built a lot of really cool technology. We learn a lot of things. I think the ideas we aren't have influence.

And systems like patrole, few people use that. I here, right? And so I think that's super cool.

And for me personally, I learned so much from IT, right? And I think a lot of the engineers work on also learned a tremendous. And so I think that um that's just really exciting to see. And you know i'm sorry that the project didn't work out. I wish I did of course, right but um but you know it's it's the research project and you're there to learn .

from IT was interest to think about uh, the evolution of programing as we come up with this whole new set of algorithms in machine learning and artificial intelligence and what's going to win out because that could be a new program language IT could be mean. We I just mentioned Julia. I think there's a lot of ideas behind Julia that module shares what what thoughts about Julian general?

Um so I would I would have to say that when we launched mojo, the one of the bigs things I didn't predict was the response from the Julia community and so um I was not I mean I i've limit take back i've known the Julia of folks for a really long time there, a dt vel of the M A long time ago. They've been pushing stay. There are a bunch different with julius, a really cool system.

Um I had always thought of julius being mostly a scientific computing focus environment, right IT. And and I thought that was its focus. Um I neglected to understand that one of their missions is to like help make python work at end.

And so I think that was my my air for not understanding that. And so i've could have and maybe more sensitive to that. But um but there's major differences between what modules doing, what july is doing.

So you say Julie is not python, right? So one of the things that about the Julie people came out and said that is like, okay, well, if we put a ton of more energy and ten more money or engineering or whatever in the Julia, maybe, uh, that would be Better than starting mojo, right? I mean, maybe that's true, but so wouldn't make july into python.

So how do you work back from the goal of let's build something for python programmer without requiring them to relearn syntax, then july just isn't there, right? Me, that's a different thing, right? And so if you anker on, i'd love Julia and I want Julia to go further than look at IT from a different ones.

But the ones coming out was everybody is using python. Python isn't syntax. I isn't broken.

Let's take what's great about python and make IT even Better. And so it's just a different starting point. So I think I think this is a great language of communities are lovely community. They're doing really cool stuff, but it's just a different slightly different angle.

But IT does seem that piton is quite sticky. Is there some uh, first of most thing you could say about why python by many measures, seems to be the most popular programming.

But I can tell you things I love about IT. Maybe that's one way to answer the question, right? So huge package go system, super lightweight and easy to digging. IT has very low .

starts up time and what .

start up time, me like you look at the line go and just takes a like java for example, takes a long time to get compile all the things and and then the vm starts up in the garden closure kicks in and then IT reves its engines and then I can tell through a lot of internet stuff or whatever, right. Um python n is like scripting because they just go right. Python is a very low compile time.

You're not sitting there are waiting. Python integrates in a new books in a very elegant way that makes exploration super and interactive and is awesome, right? Python is also um it's like almost the glue of computing because has such a simple object representation.

A lot of things plug into IT. That dynamic meta programming thing we are talking about also enables really expressive and beautiful A P S. right? So there's lots of reasons.

So you can look at technical things, the python is done and say like, okay, oh, this is actually a pretty amazing thing. And any one of those you can collect, people all just talk about invention and ignore like the fundamental things. But then you also look at the community said, right? So python owns machine learning. Machine learning .

is prety big yeah.

And it's going in important.

And there's a reputation tige to machine learning to wear. Like if you are new programmer, you are thinking about like which programming which do I use? Well, i've should probably care about machine learning. Therefore, let me try python builds and builds .

and and you even go go back before that. Like my is in python right now because i'm telling them learn python.

but because whatever telling as you or what .

they they also learn scratch things like this student. It's because python is taught everywhere, right? Because it's easy to learn, right? And because it's pervasive right there.

My, we learn to and heap less plus, but i'll pair both directions. But yes, I guess python, the main language of teaching sAlina.

if I there's these growth yle. If you look at what causes things to become popular and then gain in popularity, there's reinforcing feedback loop and things like this. And I think python has done again, the whole community is a really good job of building those growth loops and help propel the ecosystem. And I think that again, you look at what you can get down just few lines code. amazing.

So this kind of self building loop, it's interesting to understand because when you look at mojo, what is stands for some of the features, IT seems sort of clear that this is a good direction for programing languages to evolve in the machine learning community. But still not obvious that we will, because of this, whatever the engineer of popularity of reality is, there is something you can speak to like how how do you get people to switch?

yeah. Well, I mean, I think the viral growth loop is to switch people to unique. I think you could file extensions or what i'm betting on. I think that's going to be the thing.

Yeah tell the kids that you can use the fire mode .

and be like what exactly well, in in all serious ness like I mean, I think there's really i'll give you two officer answers. One is, I hope if it's useful, if that solves problems in people care about this problem, things solved, build up the attack, right? That's that's kind of simple answer.

And when you're looking to get tech adopted, the question is, is IT solving important problem, people need solved and is the adoption cost low enough they're willing to make the switching cut over and do the pain of front so they can actually do IT right? And so hopefully, mojo be that for a bunch of people and you know, people building these hybrid packages are suffering is really painful. And so I think that we have a good shot of helping people.

But the other side is like it's okay if people don't use mojo like my job to say like everybody should do this. Like i'm not saying python is bad. Like I hope python, c python like all these implementations because I not just python, is also a bunch of different implementations with different traders. And this ecosystem is really powerful and exciting um as our other programing language is not like type script or something is gna go away, right and is not a winner take all thing. And so I hope that modules exciting and use of the people, but if it's not, that's also fine.

But I also wonder what the the use case for why should try mojo would be. So practically speaking, IT seems like. So there's entertainment.

There's a dopamine e hit of sign colly shit. This is ten transfered. This little piece of all is sometimes faster .

in moger the box before .

you get something exactly I mean just even I mean that's the doping hit that um every programmer of dreams of this uh the the optimization. It's also the drug that can pull you in and have you waste way too much of your life optimizing and over optimizing. But so what would you see that would be like?

Comedy is is very hard to predict, of course. But and you look, ten years from now, modules, uh, super successful. What do you think would be the thing what people like try and then use IT regularly in the kind .

of girls and cause and so again, humans are not one thing and some people love wearing their code and arrow anything and thrown himself in the deep and and try out anything. In my experience most people don't like they are too busy. They have other things going on um by number.

Most people don't want like this I want to rewrite on my code, but even those people are too busy people. The people that uh don't actually care about the language just care about getting stuff done. Those people do like learning new things right.

And so you talk about the dope in rush of text faster. While that's cool, I won't do that again. What's also like here's here's the thing i've heard about a different demand and I don't have read on my code. I can learn a new trick, right? Well, that's called growth, me know.

And so and so one thing that I think is cool about mojo, and again, I will take a little bit of time for, for example, the blog post and the books and like all kind of soft develop in the languages get further long. But what we're doing, you talk about types like you can say, look, you can start with the world. You are no and you can progressively learn new things and adopt what makes sense if you never do that, that's cool.

You're not a bad person. You if you get really excitement and want to go all the way in the deep end and right everything and whatever, that's cool, right? I think the midst ath is actually the more likely one where it's um you know you you come out with a new a new idea and you discover a while that makes my cold wave similar way more beautiful, way faster whatever.

And I think that's what people like. Now if you fast for you said like ten years up, right I can give you a very different answer on that which is I mean, if you go back and look at what computers look like twenty years ago, every eighteen months, like a faster for free, right, to faster every eighteen months, IT was like clockwork was IT was free, right? You go back ten years ago, and we entered in this world where suddenly we had multiple or CPU and we had G P S.

If you squint and turn your head, what A G P U is just a many core, very simple CPU thing kind of right. And so um and ten years ago, IT was C P S and G P S and graphics. Today we have to push pus, graphics and air, because is so important, because the computer is so demanding, because of the smart cameras in the watches and all the different places the A I needs to work in our lives, caused the explosion of hardware.

And so part of my thesis, part of my belief of our computing goes, if you look out ten years from now, it's not going to get simpler. This isn't going back where we came from. It's only get weird from here on out, right? And so to me, the exciting part about what we're building is, is about building the universal platform, which world can continue to get weird because again, I don't think it's avoidable.

It's physics. But we can help lift people scale, do things with that. They don't even write their code every time a new device comes out. And I think that's pretty cool. And so if module can help with that problem, and I think that IT will be hopefully quite interesting, quite useful to wide range people because um there's so much potential and like there's so much you know maybe in all computers will become a thing or something, right? And we need to be able to get into a mode where we can move this programme model forward, but do so in away where we're lifting people in and growing them instead of forcing them to write the code and exploding them.

Do you think they'll be a few major libraries? The go module, first .

module. So again, come back like we're not building module because it's fun. We're building module because .

we had to dissolve these accelerated .

the organ and I mean one thon like each of the pack. Maintenance also has, i'm sure, plenty of other things going on.

People don't like really don't like read and code just for sick of the right um but sometimes like people are excited about like adopting a new idea yeah IT turns out that while rewriting code is generally not people's first thing, turns out that redesigning something while you rewrite IT and using a rewrite as an excuse redesign can lead to the two doto of your thing that way Better than the one doto. right? And so I have no idea.

I can predict that. But there's a lot of these places were, again, if you have a package that is half sea and half python, right? Just solve the pain, make IT easier to move things faster, make IT easier to bug, involve your tech adopting model, make sense to start with. Then I give you the opportunity to rethink these things.

So the two big gains are that the there's a performance game and then there's the portability to all kinds of different devices and .

there's safety, right? So you talk about real types. I am not saying this for everybody, but that's actually a pretty big thing. And and and so there's a bunch of different aspects of what what value you provide.

And so I think it's funny for me, like I ve been working on these kinds of technologies and tools for too many years now um but you look at swift, right and we touch swift tense flow, but swift as a programme language, right? Swift now thirteen years old from when I started IT. Yeah so because I started in two thousand ten, I remember.

And so that that project and I was involved with IT for twelve years or something, right? That project has gone through zone. Really interesting story, right? And it's a mature, successful, used by millions of people system, right? Certainly not dead yet, right? But but also going through that story.

C, I learned a tremendous amount about building languages, about building iles, about working with community and things like this. And so that experience, like i'm helping channel and bring jacket in a mojo. And you know other system's same thing, like apparently like building building an iteration and an evolving things.

And so you look at this L V M thing I worked on twenty years ago, and you look at M L I R, right? And so a lot of the lessons learned in l of m got fed into M L A. And I think that malaya are way Better system than a ov m was.

And know swift is a really good system. And it's it's amazing. But I hope that mojo will take the next step, step forward. In terms of a design.

uh, terms of running mojo, people can play with IT. What's the mojo playground? yeah. And from the interface perspective and from the harder or perspective, was this incredible thing running on?

yeah. So right now, so here we are, two weeks after lunch, yes, we decided that, okay, we have this incredible set of technology that we think might be good, that we have not given IT to lots of people yet. So we are very conservation and said, let's put in a workbook so if the crashes we can do something about IT, we can monitor track that right? So um again, thanks are still super early but we're having like one person a minute sign up with over seventy thousand people too. Exit is kind of crazy .

so you can sign up to the playground and you can use IT in the cloud yeah in your brothers.

And so what that's running on the yeah what that's running on is that's running on um cloud vms. And so you share a machine with a bunch of other people, but turns out there's a bunch of them now because there's a lot of people. And so what you're doing for getting free computer and you're getting play with the thing and kind of a limited controlled way so that we can make sure that IT doesn't totally crash be embarrassing. So um not a lot of the feedback we've gone as people wanted dollar around locally.

So working that right now. And so that's that's good.

I be able to downright we're going right now and we just want sure we I think this is this is one of the lessons I learned from swift. Also by the way, is that when we learn swift, uh, cash feels like forever go twenty fourteen and uh we I mean, I was super exciting. I and we the team had worked on swift for a number of years and secrecy and we uh four years into the development, roughly of work on the same.

At that point, about two hundred and fifty people at apple knew about IT. Okay, so secret. I was good at secure acy, and that was a secret project. And so we launch this.

W, we see a match of hua, an excitement, and said, developers, ers, you can be able to develop in seven after the f store, in three months. OK. Well, several interesting things to happen, right? So first of all, we learned that I had a lot of bugs.

IT was not actually production quality, and I was extremely stressful in terms of like trying to get IT working for a bunch people. And so what happens? We went from zero to.

I don't know how many developers apple head of the time, but a lot of developers overnight and they are rented a lot of bugs. And I was really embarrassing. And IT was very stressful for everybody involved, right? IT was also very exciting because everybody was excited about that.

The other thing I learned is that when that happened, roughly, every soft engineer who did not know about the project at apple there had exploded when I was launched because they didn't know is coming. And so they're like, wait, what is this I signed up to work for apple because I love objective. See why is there a new thing, right? And, uh, now what that meant practically is that the push from launch to first wall, the fob than two dot three thousand, like every all the way forward, was super painful for the engineering team and myself.

IT was very stressful. The developer community was very grumpy about IT because they were like, okay, what way you're changing and breaking my code and we have to to fix the bugs and I was just like a lot of attention and friction on all sides. Um there's a lot technical debt in the compiler because we have to run really fast.

You have to go influent the thing and unblocked the use case and do the thing and and you knows it's not right. We'd never have time to go back and do IT right? And um i'm very proud of the swifty because you've come I mean, we but they came so far and made so much progress over over this time since launch. It's pretty incredible and which is a very, very good thing. But I just don't want do that again, right?

So iterate more through the development process.

And so what we're doing is we're not launching IT when it's hopefully is at nine with no testers. We're launching IT and thank through, right. And so we're saying explications of saying like, okay, well, don't use this for production, right?

If you're interested in what we're doing, well, doing an open way and we can do IT together but don't use in production yet like i'll get there, but let's let's do where. And I also saying we're not in a race. The thing that I want to do is build the world's best thing.

Yes, right? Because if you do IT right, and that lives the industry, IT doesn't matter. IT takes next to two months. Yeah, like two months worth waiting. And so doing IT right and not being overwhelmed. Technical ad and things like this is like, again, war room um lessons learned uh whatever you want to say I think is absolutely the right thing to do even though right now people are very frustrated that know I or nothing I feature x or something like this.

What have you learned in a little bit of times since it's been released into the wild that people have been complained about future x or wires? The what have they have been complaining about what they have been uh, excited about like almost like detailed things that make I think everyone would be very excited about the big vision.

Yeah yeah so I mean i've been very pleas in fact I mean we've been massey overwhelmed with response, which is um a good problem to have um it's kind of like a success disaster yeah in a sense right um and um so I mean if you go back in time when we started modeler, which is just i'm not yet year and a half ago.

So still pretty new company, new team, small, very good team of people like we started with extreme conviction that there is a set of problems we need to solve and if we solve IT, then people be interesting what we're doing, right? But but again, you're building in basically secret, right? You're trying to figure that out.

It's the creates a messy process. You're having to go through a different understand what you want to do and how to explain IT. Often when you're doing disruptive and new kinds of things, just knowing how to explain IT is super difficult, right?

Um and so when we launched, we hope people would be excited. But you know i'm an often this, but i'm also like don't want to get ahead of myself. And so when people thought out about muji, think their heads exploded a little bit, right you know, here here's, I think, a pretty credible team that has built some languages in some tools before.

And so they have some lessons learned and are tackling some of the deep problems in the python ecosystem and giving IT the love and attention that should be getting. And I think people got very excited about that. And so if you look at that and I think people are excited about ownership and take a step beyond rust, right? There's people that are very excited about that is people that are, uh, you know just like I made game of life for four hundred times faster, right, and things like that, that's really cold. There are people there are really excited about the, okay, I really hate writing stuff hansei s plus so like this.

they are like stepping up like.

yeah so that's me by the way, I want to stop but the um I get .

third person excitement when people tweet here, meet this code game of life for whatever is faster you like .

yeah and also like what I would also say that um let me let me cast blame out to people who deserve IT sure these terrible people who convinced me to do some of this yes, Jimmy Howard, yes, that guy. Well, he's been .

pushing for this kind of .

he's been pushing.

promoting yeah.

he's wanted this for a long.

long how like the most gid people the aching learning his a as a grassroots he really teaches his incredible educator is incredible teacher, but also edit in terms of machine learning engineer himself yeah then he's been running the fast that I I and looking, I think, for exactly what .

you exactly so I mean, the first, so I met germany pretty early on. But the first time I SAT up and i'm like, this guy is ridiculous, as when I was at google and we're bring up TPU and we had a whole team of people, and where there is this competition called don bench of who can train a imagination fastest rise and german.

And one of his researchers crushed google, and not through sheer force of the amazing amount of compute in the number of tps and stuff like that, he just decided the progressive image sizing was the right way to train the model. And if few e pox faster and make the whole thing go room right? And i'm like, this guy is incredible, right? So you can say, anyways, come back to, you know, where is module coming from, Chris? Finally, listen to germany.

But is a kind of very a refreshing pragmatic view that has about machine learning that I don't know if it's this mix of desire efficiency, but ultimately ounds a desire to maker machine learning more accessible to a lot of people. I don't know that is I guess, that's coupled with efficiency and performance, but it's not just obsessed about performance.

So a lot of A I N A I research ends up being the has to go fast enough to get scale. So a lot of people don't actually care about performance particular, the research said until two thousand, to have more a bigger dataset, right? And so something now you care about, distributed computer.

And like all these exotic P C, like you don't actually want to know about that. You just want to be able to do more experiments faster and do so with bigger dataset, right? And so germany has been really pushing the limits.

And one of the things i'll save about germany, and there's many things I could have a mics on a fan boys, but uh, he a fits in his head in germany actually takes the time where many people don't. To really dive deep into why is the beta premier of the atom optimized equal to this? Yeah right. And i'll go survey and understand what are all the activation functions in the tradeoffs and why is that that everything does a you know this model pick that thing?

So the why not just trying in different values like really what is going on here.

And so as a consequence of that, like he's always he again, he makes time, but he he spends time to understand things that the depth that a lot of people don't. And as you say, he then brings that and teaches people and his his mission is to help lift, you know, his website says making A I uncool again. Like it's about like forget about the hype.

It's actually practical and useful to teach people how to do this. Right now. The problem germany struggled with, he's pushing the envelope, right? Research isn't about doing the thing that is staying on the happy path through the the will pave road, right? And so a lot of the system today, then these really fragile gile fragment things are special case in this happy path. And if you fall off the happy path, get invite already.

So what about piton? Has the genetic system of packages and is a package repository to have ideas of how to do that? Well, for module? yeah. How did your possible packages?

Well, that's another really interesting problem that I knew about, but I didn't understand how big of a problem. But was python packaging. A lot of people have very big .

pain points and lots .

cars of python packaging. You are several building and distributing .

version if you want to create your own.

yes and then or you want to build on top of a bunch of other people's packages and then they get updated this now i'm not explain this, so I don't know the answer. I think this is one the reasons why it's great that we work as a time and there's really good and smart people involved.

Um the but one of my one of the things i've heard from smart people who've done a lot of this is that the packaging becomes a huge disaster when you get the python and see together. And if you have this problem where you have code split between python ency, now not only do you have to package the c code, you have to build the c code. C doesn't have a package manager, right? C doesn't have a depends y verging management system, right? And so i'm not experience and stay of the art and uh, all the different python package managers.

But I might think that's a massive part of the problem and I think module solves that part of the problem actually had done. Now one of things I think we will do with community that is, again, we're not solving all the world problems at once. We have to be kind of focus to start with is that I think that we will have an opportunity to evaluate picture, right? And so I think that we can come back and say, okay, well, given the new tools and technologies and the cool things we have that we built up because we have not just.

syntax. We have an entirely new compiler stack that works in a new way. Maybe there is other innovative ons we can bring together and maybe we can help .

solve that problem. So almost attention to that question from the user perspective of packages IT was always surprising to me that he was not easier to sort of explore and fine packages, you know with pip installed IT IT feels uh, is an incredible ecosystem. Just a interesting there wasn't made so I think not made easier to discover packages to do like A A search and discovery as to cause IT. Well, I mean.

it's kind of funny because this is one of the chAllenges of these like intentionally delayed communities. And so I don't I don't know there are dancers for python. I mean, there are many people. I only know the right answer for mojo like so there are many people that would have much more informed experience that I do. But but it's interesting if you look at this right open source communities, um you know there's get get as a fully decentralized and I can do anyway they want, but then there's get up and get centralized commercial in that case, right thing uh really help pull together and helps out some the discovery problems and help build a more consistent community. And so maybe there are opportunities .

for for something like a get help yeah although even gip might be wrong and this but the desertion discovery, forget, was not that great I guess to use google search yeah well.

I made IT. Maybe that's because doesn't want replace google search, right? I I think there is a room for specialized solutions to specific problem. I I don't I don't know the radiance to forget hobby that they can go figure that out.

But the point is to have an interface as useful as successful to people of all different skill levels.

Well, and again, of the benefit of standards, standards allow to build these next level ecosystem, next level infrastructure and next so um again, come back of a complexity. C C A plus python is complicated. IT makes everything more difficult to deal with, that makes IT difficult port move, cut around, work with all these things get more complete. And so I mean, i'm not expert, but maybe mojo can help a little bit by helping reduce tempt to see in the ecosystem and make IT their for school.

But so you kind of packages is that hybridity nature would be a .

natural fit to move to a lot of them.

especially they are doing some interesting stuff. competition. Alist, let me ask boss some features. So we talked about obvious ly mentation, that is the type guage or option oni typed. So three, say the optional.

Progressively, progressively, I think so, so, so people have very strong paints on the right word use.

I don't know, I look forward to letters so there's the the virus is, but let is for Constance. Uh, there is an optional.

uh, yes, very, very mixable.

So you can resign OK. Uh, then there's a function overloading, oh, okay, think there is a lot of source of happiness for me, but function overloading this. Um I guess is that is that for performance or that why does python not have function so I .

can specular so um python is a dynamic language. The way that works is the um python objective c are actually very similar worlds, if you ignore in tax. And so the objective sea is straight line driver from small talk, a really vulnerable, interesting language that much of the world is forgotten about.

But the people that remember IT love IT generally um in the way that small talk works is that every object has a dictionary in IT and the dictionary maps from the name of a function or the name of value within an object to its implementation. And so the way you call a method, an object sees you say, go look up the way I call foods, I go look up food. I get a point out of the function back, and then I call IT. That's how python works, right? And so now the problem of that is that the dictionary with in a python object, all the keys are strength, and it's a dictionary, so you can only have one entry per name.

You think it's something is .

that I think it's simple as that. And so now why do they never fix this? Like why do they not change IT to not prediction? no. Like do other things.

Well, you don't really have to in python because this dynamic and so you can say I get into the function now I got has an inner do some dynamic test word if it's a string um there's another additional chAllenge which is even if he did or overloading you saying, okay, well here's a version of a function for angers and function for strings well, you'd have even if you could put IT in that dictionary, you'd have to have the color to the dispatch. And so every time you call the function, you'd have say, like isn't insure the string and so you've figured what to do that test. And so in a dynamic language, um overloading is something you generally you don't have to have.

So but now are you get into a type language and you know in python a few subscript with an image, then you get typically one element out of a collection if you subscribe with a range, you get a different thing out, right? And so often in type languages, you you want to be able to express the fact that, cool. I have different behavior depending on what I actually passed into the thing. If you can model that, I can make IT safer, more predictable and faster.

Like all these things, somehow feels it's safer, yes, but also feels empowering. Make interns of clarity, like, you know, have the design hold different functions?

Yeah this is also one of the chAllenges with the existing python typing systems. Is that in practice, like you take subscript, in practice, all of these functions, they don't have one signature. They actually have different behavior in different cases. And so this is why it's difficile to like retrofit this into existing python code and make IT play well typing. You can have to design for that.

okay. So there's a interesting distinction that people the program python might be interested in is death verse F N. So two different ways to find a function. And A F N is A A stricter version of death. What's the coolness that comes from the strictest?

So here you get into what is the trade off with the super cent? yes. So super, you have to you really want to be compatible. Like if you don't a super set, you've decided compatibility with existing code is the important thing, even if some of the decisions they made or maybe not what you choose. yeah.

So I mean, you put a lot time in comparability and that means that you get locked into decisions of the past even if they might not been a good thing right now. Systems programmer typically like to control things right, and they want to make sure that you know not not on all cases, of course, and in even systems programmer or not one thing, right, but but often you want predictability. And so one one of the things that python has, for example, as you know, is that if you to find a variable, you just say x ics for I have a variable named to x now I say some long, some long name equals seventeen.

Print out some long name hoops by type ted, right? Well, the compiler, the python compiler doesn't know no case is what you're defining what you're using and did you type the use of IT or the definition? right? And so for people coming from type languages, again, i'm not saying the right wrong, but that drives me crazy because they want to compiler to tell them you type out the name of f does as you say, it's a strict mode and so says, okay, well, you have to actually do are intentional declare variables before use them that give you more predictability, more uh, air checking and things like this. But you don't have to you don't have to use IT and this is A A way that mojo is both compatible because deaths work the same way that depth of sorry always work. But IT provides new alternative to give you more control and allows certain kinds of people that have a different flows to be able to express and get that.

But usually if you're writing major coal scratch, you're be using a fan.

IT depends again, depends on your mentality, right? It's it's not the deis python and F, N is mojo. Mojo has both and IT loves both.

IT really depends on strict. Yeah exactly. Are you are you playing around and scripting something on? Is that one or three script? cool? Like, great.

I will still be. But yes, I love.

This is a control power. You also like suffering, right? Yes, how? How many pupils .

I ve lost counted? This said at this point.

So, and that's cool. I love you for that yeah and I love other people like strict things, right? But but I don't want to say that that the the right thing because python also very beautiful for hacking around and doing stuff and research and these other cases where you .

may not want that he had just feel like, uh, maybe i'm wrong that but he feels like strictest least the faster to bugging. Some terms of going from even on a small project from zero to completion is just the, I guess, IT depends how many bugs you generate usually.

Well, so I mean, it's again, lessons learned looking at the ecosystem. It's really I think it's if you study some of these languages over time, like the ruby community, for example, now ruby is a pretty well developed, pretty sabic community, but along their path that they really invested in unit testing.

I think that the rubi communities really pushed forward stay at the art of testing because I I didn't have a type system that cut a lot of bugs to compile time, right? And so you can have the best ball throws, you can have your testing and good types. So and things like this, I thought that was really interesting to see how certain chAllenges gets solved, ed.

And in python, for example, the interactive note of kind of experiences. And so like this, really amazing. And if you type of something that doesn't matter, IT just tells you what's fine, right? So I think that the trails are very different if you're building a um in large cae production system versus your building in expLoring a notebook and .

the speaking out of control the Hillary ous thing. If you look at code I regions for myself for fun, it's like littered with a search everywhere. It's a kind of.

yeah.

you'd like this is basically saying, in a dictator way, this should be true now, otherwise everything stops.

That is the sign I love you, man. That is a sign of someone who likes control yeah and so yes, I think you like a fan. You I think there .

be session. Yes, I definitely. Well, uh, speaking of the search, exceptions are called the eras.

Why is IT called aas? So I mean, we the same were the same. right? But um we implemented very different way, right? So if you look at um other languages like will put on sea plus plus or favorite, right c POS plus has a thing called zero cost exception. Hy so this is in my opinion, something to learn .

lessons from. It's a and so um and .

so zero cost exception handling the way that works is that it's called zero cost because if you don't throw an exception, there's supposed to be no overhead for the non error code. And so IT takes the air path out of the, the common path。 Um IT does this by making throwing an air extremely expensive.

And so if you actually thrown air with a simple plus compiler using exceptions has to look up in tables on side and do all the stuff throwing and air could be like ten thousand times more expensive than referring for a function right um also is called the access exceptions but is not zero cost by any stretch imagination because IT masters blows out your code nearby. Now IT also adds a whole bunch of different path because of structure and other things like exists to all these effects. And so the saying that was called dorcas exceptions IT really ate okay.

Now if you fast ward to to neural languages and um this includes swift and rust and go and now mojo um uh well in python little bit different because it's interpreted and so like it's got a little bit of different thing going. If you look at IT, if you look at compiled languages, many newer languages say, okay, well let's not do that zero cost exception handling thing. Let's actually treat and throwing the air the same as returning a variant, returning either the Normal or or near.

Programmer generally don't want to deal with all the typing machinery and like pushing around a variant. And so you use all the syntax the python gives us, for example, try and catch and you know, functions that raise and things like this. You can put a raises decorator on your functions, subject this and if you want control that and then um the language can provide syntax or but under the hood, the way the computer executes throwing areas basically .

faces returning something something.

And so this is actually, I mean, it's it's a fairly nerdy thing, right because I love IT um but this has a huge impact on the way design A P S, right. So in simple plus huge communities turn off exceptions because the costs is so high, right? And so the zero cost costs so high, right? And so that means you can actually use exceptions in many library aries.

And even for the people .

they do use IT. Well, okay, how and when do you want to pay the cost if I try to open a file so I throw an error? Well, what if i'm pRobing around looking for something right? I'm looking up in matter from past.

Well, it's really slow to do that. May be had another function that doesn't turn air returns an arco instead. And I have two different versions, the same thing. And so IT causes you to forker p and so, you know, one of the things I learned from apple and I so love is the art of API design is actually really profound.

I think this is something that python also done a pretty good job at in front of building out the article e package assist m about having standards and things like this. And so you know we wouldn't to enter a mode where um you know there's a thereof feature the existent language, but people don't use practice. Now also say one of the other really cool things about this implementation approaches that I can on G, P.

U. And I can on accelleration ors and things like that. And that standard zero cos exception thing would never work on an auditor. And this is also part of how can scale all the way down to, like, little embedded stems to run entries es and things like that.

Can you actually say about the maybe is there some high level way to describe the chAllenge of exceptions and how they work in code during completion? So just this idea of calculating up a thing, an error.

Yeah, yeah. So the way the way to think about is um think about the functions that doesn't return anything, just is a simple case, right? So you have function, one calls function, two calls function, three calls function.

For along that cause stack there are try blocks, right? And so if you have function, one calls function to, function to as a tribal ck. Then was then a IT calls function through right.

What happens to function? Three throws? Well, actually start simpler. What happens for returns? Well, the reasons this, please go check out and continue executing, and then fall off the bottom, the try block and keep going as good if the function throws you supose to exit the current function, and then get into the accept class, right, and then do whatever course, sir, then keep falling on and going on.

And so the way that I compile, like major works is the call that function, which happens in the except block calls the function. And then instead of returning nothing, IT actually returns, you know, an a variant between nothing and an error. And so if you return Normally for the bottom to return, you return nothing.

If you throw throw air, you return the very thing that is, i'm an err, right? So when you get to the call you say, okay, cool, I call the function. Hey, I know locally i'm in a tribal ck, right? So I call the function. And then I check to see what IT returns a ha is that everything jumped to the except work.

and that's all down for you behind the scenes.

exactly. And so the computer is all this for you. And I mean, one of the things, if you dig in to have the stuff fooks and path on, because little bit we're complicated because you have finally blocks which now you need to go into do some stuff and then those can .

also throw .

and return like the stuff matter of ability um like there's no there's with clauses and so with causes are kind of like finally blocks some special stuff going on. And so there nothing in general.

nothing of anything, nothing of functions should be illegal. Well, IT just feels like that as a level complexity.

I like a million implementation. And so this is again one of the one of the tradeoffs you get when you'd say about the super site is you get the influence of fulfill implementation of the thing that you decided is good. And so yeah, I mean, we can we can complain about the reality, the world shaker fist. But IT .

always feels like you shouldn't a lot to do that like to declare .

function, inside functions.

inside functions. The lip, lip is what I used to do .

in college. So you've grown up.

We've all done things and cause of my part. Now I ve OK yeah going to say.

you're afraid of me.

You're taking, I love IT s IT worked as a joke on my head and ah right .

nessa's functions are joke inside, actually really great and for certain things, right? And so these are also called closures. Closures are prety cool, then you can pass call back. S there's a lot of patterns.

And so h so speaking, which I don't think you have uh, nested functions implemented .

yet in mojo, and we don't have limb to some there.

A few things on the road map that, that be cool to just fly through because it's interesting thing. You know how many features there are no language, small and big. Yes, they have to implement yeah the first others to support and that has to do with some very specific aspect of IT like the parents asy or not printing .

asia that just a totally a .

syntactic thing, synthetic thing. Okay, there's but it's cool still a so keyword arguments .

and functions yeah so this is where in python you can say color .

function x equals four yes yeah I mean.

again, this is an rocket sense experiment .

is just the longest sist um the bigger .

features of things like traits. So traits are, when you wanted, define abstract. So when you get into typed languages, you need the ability to write generic.

And so you want, say, I want to write this function, and I want to work on all things that are arithmetic. Like, well, what is a arithmetic like mean? Well, arithmetic like is a categorization of a bunch of types.

And so again, you can to find a many different ways. And i'm not going to going to ring through or something, but the you know you can say something like like if you can add the track most way to buy IT, for example, right. And so what you're saying, just saying there's a set of traits that apply to broad variety caves.

And so there all these types of arithmetic, like all these sensors and flooding point injured, like there's this category of types. And then I can define on an authority, al access algorithms that then work against types that have those properties. And so this is a again, it's a widely known thing. It's been in plant and swift and rest and many languages. So not um which is where everybody learns learn to the next from um but the but we need temple that that will enable a new level of expressivity.

So classes yeah .

classes .

big a big deals are still to be implemented like you said, a ladson tax. And that is like detail stuff I call my import support for top level code fell scope, so and then global verso. So being able to have variables outside of the top.

And so this construct that where module came from and the fact this is a point one, right? And so we're building the module is building A S stack. And A S stack has a bunch of problems working with hardware and writing.

Hy performs corneal and doing with cornal fusion thing I talking about and getting the most out of the hardware. And so we ve really prioritize and built more, solve much problem right now. Our our stars built out and support all the things. And so we're making incredible progress, by the way, emerges only like seven months old. Another interesting I mean.

part of the reason I want to mention some of these things is like there's a lot to to do and it's pretty cool. You just kind sometimes you take for grand how much there is in the programme language, how many cool features you kind of lie on and this is kind of a nice reminder late as to do this.

yeah. So I mean, but also you look into it's it's amazing how much is also there and you take him for grand that um a value if you define IT IT will get destroy automatically. Like that little feature itself has actually really complicated given the way the ownership system has to work and the way that works with modules. A huge step forward from a rose and swift of.

again, value you defining, destroyed.

whatever that means, like in your local function, right? And so you say, I like whether being a death and so you just say, x equals. Hello, world, right? Well, if your string type requires you to allocate memory, then once to throw row, you have to delicate IT.

So in python and mother you to find that with the dell method, right, where is that IT run? Well, IT gets run sometime between the use, the value and the end of the program like this. You now get into a garbage collection.

You get to like all these long debate, you talk about religions and and trade off some things like this. This is a hugely hotly contested world. If you look at the police, plus the way this works is that if you to find a variables or a set of variables within a function, they get destroyed in a last in first out order.

It's like nesting. Um this has a huge problem because if you to find of a big scope and you find much at the top and then you use them and then you do a whole bunch of code that doesn't use them, they don't get destroy until the very end of that scope, right? And so also of trace tail calls.

So good functional programing, right? This has a bunch of different impacts on um you know you talk about reference counting organizations and things like this, a bunch of very low level things. And so what motor does has a different approach on that from any don't famine with where destroy them as soon as possible.

And by doing that, you get Better memory use, you get Better predictor, you get tail calls that work, you get a bunch of other things, you get Better earner's tracking. There's a bunch of these very simple things that are very fundamental. There are we built in their mother today that are the things that nobody talks about generally. But when they don't work right, you find out and you have to complain about .

is a trivial to know, uh, what's as soon as possible to delete a thing that is not going to be used again?

Yeah I mean, it's generally trivial. It's after the last year of IT. So if you find X A string and then you have some use of x summer.

your code within the scope an within the scope accessible.

It's yeah exactly. You can only use something within its scope. And so then IT doesn't wait until the end of the scope to delete IT. IT destroys IT after last years.

So there's kind of some very eager machine is just sitting there and deleting.

It's all the compilers not also go. And so yeah and so what in this is actually not trivial because you have control flow. So it's kind in pretty quickly. And so like right .

was not to insert delta, like in a lot of .

places potentially yeah about the reason about this. And again, this experiences building languages are not going right. So again, you get another change to do IT.

You get basic things like this, right? But it's it's extremely powerful when you do that, right? And so there's a bunch of things like that, that kind of combine together. And this comes back to the you get a chance to do the right way, do IT the right way and make sure that every brick put down is really good, so that when you put more bricks on top of that, they stack up to something is beautiful.

But there's also a company design discussions. Do there have to be about particular details like implementations of particular small features? Because the features that seems small, I bet some of them might be like really a require really big design decisions?

yes. Well, so I mean, let me give another example. This python has a feature called a away, so it's a new feature. I mean in in the long ger on history it's relatives vely new feature, right that allows way more expressive a synchronous programme. Again, this is this is a pite the beautiful thing and they do things are great for mojo for completely different reasons.

Um the reason that I think wake added to python, as far as I know, is because python doesn't support threats and so python doesn't support threads. But you want to work with networking and other things like that, that can block I mean path on does support threads is just not a strength. And so and so they added this feature called they think away.

It's also seen other languages like swift and java script and in many other places as well. Um I think weight and modules amazing because we have a high performs heroes computer runtime underneath the covers that then allows non blocking I O so you get full use of your accelerator. That's huge.

Turns out it's actually really important part of the utilizing the machine. You talk about design discussions that took a lot of discussions, right? And IT probably will require more iteration. And so h my floss with mother is we have A A small team, really good people that are pushing forward and they're very good at the extremely deep knowing how the compiler and run time and like all the the low level stuff works together um but they're not perfect the same thing as fifteen, right?

This is where one of the reasons we released module much earlier and so we can get feedback and we've already like rename the keyword, did a community feedback and sure uh, we is an understand and now it's named in. Now we're we're not renewing existing by thanky words because that breaks compatibility. Naming things were adding and making sure that they are designed well, we get usage experience.

We iterate work with community. Because again, if you scale something really fascinated, right though the Collins are using a in production, then it's impossible to change. And so you want to learn from people. You want to try and work on that early on. And this is where design .

sons is important. Ate into the language, the main language.

like a like favorite.

really like interesting human, like lawful, whatever, ruling on the floor, laughing. So that could be like, who would that be the use case for that? Like an expect thrown exception on some .

sort I file feature request.

Uh, maybe a heart one. IT has to be a heart one.

People told me that i'm insatiable. This is like this. I am going .

to use the viral nature of the internet to actually get this to get this past .

and is funny. Come back to the flame log file exception right the um you know we have the option to use the flame I which just even that concept because for example, the people get upset. Now i've seen everything yeah there's something .

kind is reinvigorating like it's like all that's possible that's really cool that that for some reason that makes everything else lexi exciting .

is ready for this stuff, right? So when we have a package manager will clearly have to innovate by having the compiled package to be the little box with the boon IT, right? I mean, IT has to be done.

IT has been done. Is there some stuff on the road map of the year particularly stressed about or excited about that? You think in about a lot.

I mean, as a today's snapshot, we shall be obviously tomorrow. Uh, the lifetime stuff is really exciting. And so lifetime give you safe references to memory without danger.

Ling pointings, and this has been done in language like cross before. And so we have a new approach, which is really cool. I'm very excited about that.

That would be how to community very soon. Um the traits feature is really a big deal. And so that's blocking a lot of E P, I design.

And so there is that I think that's really exciting. Um a lot of IT is this kind of table stakes features. Um one of the things that is uh again also lessons learned with swift uh is that uh programmer in general like syntactic sugar.

And so it's like, oh well, this annoying thing like like python you have just felt and and where d why can I just use plus def plus? Come on, why can I just do that? right? And so trial bit syntactic sugar IT makes sense. It's beautiful. It's obvious we're trying not to do that.

And so um for two different reasons, one of which is that I can less on a swift which has a lot of and tactic sugar um which may maybe the thing maybe not I don't know, but um but because it's such an easy addictive thing to do sugar like make sure look get crazy right um like the community will really dig into that and want to do a lot of that. And I think it's very distracting from building the correct actions. Second, we want to be good member of the python community, right? And so we want to work with the broader python community.

And yeah, we're pushing forward a bunch of assistant programme features, and we need to build them out to understand them. But once we get along with forward, I want to make sure we go back to the OK. Let's do some design reviews, talk about you how we want this stuff all to work together. And syntactic sugar just makes all that more complicated. So and yeah.

this comprehensions are yet to be impact. And my favorite, I mean, dictionaries.

point one. One is.

as you mentioned, modeler is very new. Mother is very new. It's a relatively small team.

Yeah, this building up this gigantic stack, this incredible stack, is going to perhaps to find the future of development of R A. I. overlords.

We just hope that will be useful.

as do all of us. Uh, so what what have you learned from this process of building up a team? Maybe one question is, how do you hire yeah great programmers, great the people that Operate in this compiler hardware, machine learning software, interface design space, and maybe look very fluid yeah what they can do. So, okay, so language design to so .

building a company is just as interesting in different ways is building a language like different skill set, different things but super interesting. And I ve put a lot of teams a lot from places um if he's zoom in from the big problem into recruiting well. So here's our problem.

But i'll just to be very strict about this. We started module with a lot of conviction about we understand the problems, we understand the customer pain points. We need to work backwards from the suffering in the industry. And if we solve those problems, we think of useful for people.

But the promise is that the people we need higher, as you say, are all these super specialized people that have jobs at big tech, big step goals, right? And you know we I don't think we have um park market fit in the way that a Normal start up does. We don't have park market market fit chAllenges because right now, average using A I and so many people are suffering and they want help.

And so again, we start with strong confection. Now again, you have to hire and recruit the best, and the best all have jobs. And so what we've done, we said, okay, well, let's build an amazing culture.

Start with that. That's usually not something a company starts with. Usually you hire a bunch of people.

People start fighting and IT tranto jack act mess. And then he tried to figure out how to improve your culture later. Um my co founder and particular supervision about making sure that that's right. And we've spent a lot of time early on to make sure that we can scale.

I can you come to say before, get to the second, yeah, what makes for a good culture?

So I mean, there's made different cultures. And I have learned many things from my work.

several very unique, almost famously ne cultures.

And some of them I learned what to do and something I learned what not to do. So and so um we want an inclusive culture. Uh, I believe in like amazing people working together.

And so I see in cultures where people you have amazing people and they're fighting each other, I see amazing people and they're told what to do, like double shot lined up and do what I say IT doesn't matter for the right thing, do IT right? And neither that this is. And i've seen people that have no direction, they're just kind of flooding in different places, and they want to be amazing this to how.

And so a lot of IT starts with have a clear vision. So we have a clear vision what we're doing. And so I kind of grew up at apple in my engineering life, right? And so a lot of the apple DNA roped off on me.

A Michael fantin also is like a strong protocol. And so what we learned is I started apple that you don't work from building cool technology. You don't work from like come up with the cool product and think about the features you'll have in the big checks, xing and stuff like this because if you go talk to customers, they will actually care about your product. They don't care about your technology.

What they care about is their problems, right? And if your product can help solve their problems, well, hey, they might be interesting that, right? And so if you speak them about the problems, if you understand and you have compassion, you understand what people are working with, then you can work backwards to building an amazing product, the vision, and then can the problem, and then you can work backwards and solving technology.

And an apple like this, I think pretty famously said that you know for every you know there is one hundred notes, every yes, I want to find that to say that there's one hundred not yet forever yes. But um famously if you go to the iphone, for example, right I phone one read I mean many people that IT because I didn't have three G I didn't have copy and paste right. And then a year later, okay, finally IT has three g but still doesn't have copy in paces.

It's a joke. Nobody will ever use this proof ba ba ba ba ba right. Year three, I had copy paste and people stop talking about right and so and so being laser focus on having conviction, understanding what the core problems are and giving the team the space to be able to build the right tech is really important.

Um also I mean, come back to recruiting, you have pay well, right? So we have to pay industry in sales, have good benefits and things like this. That's a big peace, a remote first company.

And so we have to. So remote first has a very strong set, poison coins. On the one hand, you can hire people from wherever they are and you can track amazing talent even if they live in strange places or unusual places.

Um on the other hand, you have time zones. On the other hand, you have like everybody on the internet will fight if they don't understand each other. And so we've had to learn how to I have a system where we actually fly people in and we can hold something together periodical, and then we get all groups together, and we plan an execute to .

get like an inmate y to the in person brainstorming. Yes, I guess you lose, but maybe don't. Maybe if you get no other well and you trust each other.

maybe you can do that when the pencil first said, I mean, i'm about your problem to the first thing I missed was having White boards. Yeah, right? Those design discussions really like I can high intensity work through things, get things done, work through the problem of the day, understand we are, figure out to solve the problem, move forward.

Yeah, but we figured out ways to work around that. Now you know all these uh, screen sharing, other things like that we do the thing is now is sitting down to lunch table with the team and the spontaneous things like the the the coffee, the coffee bar things and the and the bumping into each other and getting to know people outside of the transaction. Solve a problem over sume.

And I think there's there's just a lot of stuff that are not an expert this I don't know who is. Hopefully there are some people but that stuff that somehow missing on zoom, even with White board, if you look at that, if you have a room with one person at the White board and there is like three other people at the table. There's first though, there's a social aspect to that where you're just shooting to share a little bit, almost like .

people just kind of coming .

in yeah that but also while that like it's a break out discussion that happens for like seconds at a time, maybe inside joke like this interesting dynamic that happens. The zoo, your bonding, your bonding, your bonding. But through that bonding you get the excitement of certain ideas are like complete bushi and you'll see that in the faces of others that you won't seem necessary. And zoo m like something that feels like that should be possible to do without being in person.

Well, I mean, being in person is a very different thing. Yeah, I don't. It's worth IT, but you can always do IT. And so again, we're still learning and we're also learning is like humanity with this new reality, right? But um but what we found is that getting people together, whether be a team or the whole company, whatever, is worth the expense because people work together and happier after that. Like I just I just like there's a massive AR timey like go out and things start going for IT people together and then you realize that all working together, we see things the same way we work through the disagreement of the misunderstanding we're talking across sy other, you work much Better together. And so things like that, I think you're really quite important.

What about people that are can specialize in very different aspects of the stack together? Interesting chAllenges.

So mean, there is interesting. Tell him you know how to deal with you. But one of .

the most lovable.

And so one of the so there's different locations in building teams a for me and so some people say higher ten nex programmer and that's the only thing whatever that means, right? Um what I believe in is building well bounds teams, teams that have people that are different in them like we have all generals and no troops, all troops in no generals, or you have all people that think in one way and not the other way.

What you get is you get a very biased ed and cute d and weird situation where people end up being unhappy. And so what I like to do is I like to build teams of people where they're not all the same. No, we do have teams and their focus on like run time or compiler gp works, whatever the speciality is, but people bring a different take and have a different perspective.

And I look for people to compliment each other. And particularly if you look at leadership teams and things like this, you don't want everybody thinking the same way. You want people bring different electives and experiences. I think that's really important.

That's team. But what about building A A company's ambitious module? So what interesting things .

I mean so many like so um one of the things I love about, okay so much the first company I built from scratch um a one of the first things was profound was i'm not cleaning up somebody else this mess right and so if you look at let's liberating, just not good, it's super liberating. And um and also many of the projects i've built in the past have not been called the product to the company.

Swift is not apple's product, right? Uh M L A R is not google revenue machine or whatever, right? It's not it's it's important, but it's like working on the accounting software for you know the retail join or something, right?

It's like enabling infrastructure and technology. And so modular, the the tech we're building is here to solve people's problems like IT is directly the thing we're giving the people. And so this is a really big difference.

And what that means for me is a leader, but also for many of our engineers, they're on the thing that matters. And that's actually pretty I mean, again, for for computer people and think like that, that's usually not the case, right? And so that's that's also pretty exciting and and quite nice.

But the um one of the ways that this manifesto that makes easier to make decisions. And so one of the change I ve had in other world, this is like, okay, well, community matters somehow for the goodness of the world, like, or open source matters, therefore. Ally, but I don't want to pay for A T shirt, a right, or some slagg.

Like, well, t shirt cost ten box is you can have hundred t shirts for thousand dollars to a mega core, but thousand models. Accountability can count that low, right? But justifying in getting t shirt, by the way, if you like t shirt.

I would one hundred percent like A T shirt. Are you joking?

A fire magic t shirt. The.

I will, I will treasure this. I will pass IT down in my grandchildren .

and so it's it's very liberating. I think I should have A T shirt right and IT becomes very simple because I .

like likes this is awesome. Um so I have to ask you about the one of the interesting developments with large language models is that they're able to generate cold recently really well yes to degree that um maybe A I don't know you understand but I have I struggle to understand because I IT forces me to ask questions about the nature programing of the nature thought because the language models to predict the kind of code was about the right.

So well that makes me wonder, like how unique my brain is and where the valuable ideas as you come from. Like how much do I contribute in terms of, uh, ingenuity, innovation to code I write or design in that kind of stuff? When you stand on the shoulders of Janette, you really doing anything and what alarms are helping you do, they help you stand on the shows, just program.

There's mistakes. They're interesting that you learn from them. I just I would love to get your opinion first. High level of what you think about this impact of largely language models when they do programs and this is when they generate code.

Yeah well so um I don't know where IT all goes yeah um i'm an optimist. I'm a human optimist. I think that things i've seen or that a lot of the elms are really good at crushing liquid projects and they can reverse the link.

This like crazy one. Turns out there's a lot of instances of that on the internet and a Price stock then. And so if you want to see standard questions answered, I can memorize all the answers that can be amazing.

And also, they do generalize up from that. And so there is good work on that. But but I think that if you, in my experience, building things, building something like you talk about module, you talk about these things, you talk about building apply solution to a problem, it's also about working with people.

It's about understand the problem. What is the product that you want to build? Where are the use case? Are the customers can just go survey all the customers because it'll tell you that they want a faster horse, maybe they need to.

Cur, right. And so a lot of IT comes into um I don't feel like we have to compete with alums. I think they'll help automate a ton of the mechanical stuff out of the way. And just like now, I think we all try to scale through delegation and things like this, delegating wrote things to an alem, I think is extremely viable, uh, approach that will help us all scale and be more productive.

I think it's a it's a fascinating companion.

but I D say I don't think that means that we're can be done with coding.

but there's power in IT as a companion. And from there I could I would lost us women on to module a little bit. Do you think um do you think about that? Do you think about alms generating moto code um and helping sort of like when you decide new programing language IT almost seems like man would be nice sort of um almost is a way to learn how suppose to use this thing for them to be trained on some of the most good.

So I do lead the accompany so maybe there will be a mojo l at some point um but if you're question is like how do we make a language to be suitable for alms? Yeah I think that the um I think the cool thing about alums that you don't have to right? And so if you look at what is english, or any, these other terrible languages that we as humans deal on a continuous spaces, they are never designed for machines.

And yet there are the intermediate representation, there are the exchange format that that we humans used to get stuff done right. And so this programing languages there, an immediate resented between the human and the computer, or the human in the compiler, roughly, right? And so I think the elms will have no problem .

learning whatever key.

What do we pick?

Maybe the five actually enable IT. One of the issues I can see with being a super set of python is there be confusion by the great area. So be mixing stuff. Uh.

but i'm a human optimism. And often, as I think that will solve, you look at, okay, reducing the right thing, right? IT turns out compiles of very particular and they really want things, they really want the invention to be right.

They want there on your s in right. I mean compulsion can do Better at this, but um alliance can totally help solve that problem. And so i'm very happy about the new a predictive coding and copilot features and things like this because I think I will just make us more productive.

It's still messy and fussy and uncertain, unpredictable. So but is there future you see, given how big of a leap GPT four was where you start to see something like columns inside a compiler or no ah I mean.

you could do that absolutely I think be as I wise well. Well, I mean, nobody is very expensive, so compellers run fast. They're very fictional.

Ent and EMS are renting very expensive. There's on device salams and there's other things going on. And so maybe there's an answered there. Um I think that one of the things that I haven't seen enough of is that so elms, to me, are amazing when you happen to the creative potential of the hlubis ation, right? And if you're build doing creative brainstorming or create writing or things like that, the holus instance working in your favor and if your writing code has be correct because you're going to ship in production, then maybe that's not actually a feature.

And so I think that their husband research and their husband work on building algic reasoning systems and kind of like figuring out more things that feel like proofs. And so I think that there could be interesting work in terms of building more reliable scale systems. And that could be interesting. But if you have chased that robot, hold down, the question then becomes, how do you express your your intended the machine? And so maybe want L, M to provide the spec, but you have a different kind of that, that actually implemented the code.

So to use this documentary and and inspiration verses the .

actual implementation.

since a successful modeler will be the thing that runs. I say so joking we are A I overlords but A I systems that I used across and what negotiate term but um in a of things so costs so so .

I can say like A G I should be written in module A G I should .

moger you're joking but is also possible that is not a joke um that a lot of the ideas behind modules um seems like the natural set of ideas that would enable add scale, training and inference of AI systems um so just have to ask about the big four sopo question about human civilian. So folks like uh allia a really concerned about the threat of the eye. I do you think about the the good and the bad that can happen at scale deployment of air systems?

I've thought a lot about IT and there is a lot of different parts of this problem, everything from job displacement to sky night. Thanks like this. And so you can zoom into some parts of this problem.

I'm not super optimistic about A G I. Being solved next year. I don't think that can happen personally.

Do you have a kind then like calm about is there is a nervousness because the leap of GPT for seem so big.

sure. Like where .

almost were the some kind of transition europe you're thinking. So I mean.

there's a couple of things going on there. One is um i'm sure GPT five and seven and nineteen will be also huge leaves. They're also getting much more expensive to run.

And so there may be a limiting function terms of just expense on one hand and train like that, that could be a limited that slow things down. But I think the bigger limit is the outside of like sky that takes over. And I don't i'm thinking about that because the over so I don't know about so you know I mean, that is okay.

Other things were about i'll just focus on yeah i'll focus not really about that one um but I think that the the other thing to say is that A M is quickly, but humans move slowly and we adapt slowly. And so what I expect to happen is just like any technology to fusion, like the the promise and then the application takes time to roll out. And so I think that um i'm not even too worried about anthon's cars defining away all the taxi drivers. Remember, autonomous will be .

solved twenty.

twenty and I boy do IT so and and so like I think that on one hand, we can see amazing progress, but on the other hand, we can see that, uh, you know, the reality is a little bit more complicated and IT may take longer to roll out them.

Then you might expect, well, that's in the physical space. I I do think in the digital spaces, the stuff is built on top of elms that runs, you know, the millions of apps that could be built on top of them, and they could be run on millions of devices, millions of types of devices. I just think that the rapid effect, or hasn't human .

civilization .

could be truly just for .

predict. And I think that depends on our your optimistic pessimist or massic.

Just clarify .

a optimistic .

human socialization.

Me too. And so I look at that as saying, okay, cool, what A I do, right? And some people say, oh, god is going to stress all, how do we prevent that? I I can look at IT from it's going unlocked.

So right, you talk about coating is going to make, I don't have do all the pentax stuff. Well, sudenly, that's a very optimistic way to look at. And you look at what a lot of a lot of these technologies have done to improve our lives, and I want that to go faster.

Would you think the future programme looks like in the next ten, twenty, thirty, fifty years? The lambs and uh with with module with modular, like the vision for devices, the hardware is compiled to the, to the different stacks of software.

So what I want, I mean in coming back to my archie's sis, right? It's complexity, right? So again, me being the optimal st, if we drive down complexity, we can make these tools, these technologies, these who hard's accessible to way more people, right? So i'd love to see is more personalized experiences, more things, the research getting into production and that being lost in europe, right? And so like the these things that impact people's lives by entering products.

And so one of the things that i'm a little bit concerned about is right now, um the big companies are investing huge amounts of money and are driving the top line of A I capture for really quickly. But if IT means that you have to have a hundred million dollars to train a model or more hundred billion dollars, well, that's going to make a very concentrated, with very few people in the world that can actually do this stuff. I would much rather see lots of people across the industries be able to participate.

Use this right? You look at this, you know, I mean, a lot of great research has been done in the health world looking at like detecting pathologies and doing radiology with and like doing all these things. Well, the problem today is to deploy until these systems, you you have to be an expert radiology and expert in AI. And if we can break down the barriers so that more people can use AI techniques and more like programing python, which roughly everybody can do if they want to, right, then I think that will get a lot more practical applications. These techniques and a lot more nature, cool but narrower demands, I think that that can be really cool.

You think you will have more or less programmer in the world than now?

Well, um I think more more programmer, but they may not consider themselves to be programmer.

be a different name for right?

I mean, you consider somebody uses, I think that arguably the most popular programing language is excel.

Yeah, right?

yeah. And so do they consider themselves with programmes? Maybe not. I mean, someone make crazy cros and stuff like that.

But but what what do you mention? Steve jobs is, is the bicycle for the mind, the logic faster, right? And so I think that as we look forward, right, what is A I I look at IT is hopefully a new programme, paradise. It's like object or new programme, right? If you want write a cats to texture, you don't use four loops.

Turns out that's not the right tool for the job, right? And so right now, unfortunately, because I mean, it's not unfortunate, but it's just kind of where things a is is weird, different thing is not integrate into programing languages and Normal tool chains and all the technologies, really weird and doesn't work. And if the baby said IT and every time is hard, words different shouldn't be that way. When you change that, when you fix that, suddenly, again, the tools, technologies can be a way easier to use. You can start using them for many more things so that that's .

why would be excited about what kind of advice could you get to somebody in high school right now or maybe early college whose curious about programing and feeling like the world is changing really quickly here? Yeah what kind of stuff to learn, what kind of stuff to work on should they finish college, they go work at a company that they are build a thing. What do you think or .

so I mean one of things that is that um you'll be most successful for work on something you're excited by. And so I don't get the book and read the book covered the cover and study and memories on our site and flash card and go build something like goes all the problem. Go build the thing that you want to exit.

Go build an APP go train model like go build something that actually is IT and set to go for yourself. If you do that, then no, there's a successor. The gentle rush is the achievement.

There's the unmoved that I think is where you know if you keep setting goals and you keep doing things and building things, learning by building is really powerful. Um in terms of career advice, I mean every is difference. Very hard to give journalizing advice um all our speaker is you know a compiler erd if every is going left, sometimes it's pretty cool.

Go right. Yeah and so just because everyone y's doing a thing IT doesn't mean you have to do the same thing. And for the heard, in fact, I think that sometimes the most exciting past your life lead to being curious about things that nobody else actually focuses on and turns out that understanding deeply parts of the problem that people want to take for granted makes you extremely valuable and specialized in ways that the heart is not. And so again, I mean, there's lots of room for specialization, lots of room for uh, journalism resets of room for different kinds in parts of the problem. But I think that is, you know, just because everything, every boy's doing one thing doesn't.

you should necessary do IT. And now the heart is using python. So if you want to be a rebel, go check out mojo and help china as the world fight the arch nemesis of complexity.

Because simple is beautiful there .

go because the possible person you've you've been so kind to me ever since have been extremely support for. Thank you for who you are for being left for being kind for fighting this um really interesting problem of how to make A I accessible to huge number people, huge number of devices.

No well so actually a pretty al person too right. And so I think that you one of the funny things about you is that besides being curious and pretty damn smart, you're actually willing to push on things and you I think that you've got an agenda like make the world think, which I think is a pretty good agenda.

is a pretty good one. Thank you so much .

for talking like yeah thanks .

for listening to this conversation with Chris ladner. To support this bag gift, please check out our sponsors in the description. And now let me review some words.

Rise a gama. I do not fear computers. I fear the lack of them. Thank you for listening and hope to see you next time.