Gwern Branwen - How an Anonymous Researcher Predicted AI's Trajectory
01:36:43 Share

Today i'm interviewing gun brand. Wynn is an anonymous internet researcher and writer. He is deeply influences the people who are building A G.

I. Who was on the first people to see L M. Scaling coming.

If you read this blog, you know he's one of the most interesting polymax makers alive. We record this conversation in person in order to protect grants. Anim ity recreated this avatar. This isn't his voice, this isn't his face, but these are his words. Warn, what is the most underrated benefit of anonymity?

I think the most underused benefit of anonymity is that people don't project on to you as much. They can can't like slot you into any nh identity and like and Operating you off in advance. You know every everyone has to read you at least a little bit to even begin to dismiss you.

It's great that people can retell you against you. And i've derived a lot of benefit from people not being able to, like male hair, win to my home and called the police to swap me. But but I always feel that the biggest benefit is just that you get to hearing IT all basically, you don't get immediately written by the context.

Do you expect companies to get automated top down starting with the C E O or from the bottom up, starting with workers?

All the pressures, I think, or to go bottom up um and from existing things, it's just much more palatable every way to start at the bottom and replace there and then work your way up um to eventually kind of just having human executives overseeing a firm of their eyes.

And also from an oral perspective, I think if we are in fact Better than A S in some way IT should be in the long term vision thing, right? Like the I will be to biotype to execute any kind of novel long term strategy and seize new opportunities. So that would presumably give you this paradigm you have like a human CEO who does the vision thing and then the AI CoOperation kind like scaries around underneath them doing the C O S.

Bidding, right? And they don't have the taste that the C. E. O has. So you have one kind of Steve jobs figure at the home, and then maybe whole period of ais out there, executing the vision and bringing him new proposals.

And he, you know, he looks at every individual thing and says, no, like that proposals bad. This one is good. That may be hard to qualify, but I think that human lead firms should know under this view and of outcompeting, the entirely AI firms which were keep making these biotech choices that just don't quite work out the long term.

What is the last thing that you think you personally will be doing before your last key store is automated?

The last thing that I see myself still doing right before the box start eating about but screaming no I specifically requested the opposite of this um is I think rape before that I think what i'm still doing is the Steve jobs kind of thing of choosing um so my AI minions are like bringing me wonderful essays and i'm saying this one is Better. This is the one that I like and possibly building on that and saying, that's almost right. But you you know what would make IT really good if .

you pushed IT to eleven in this way if you do have firms that are made up of a what do you expect the united selection to be with individual as a whole? I mean, with humans we have these debates about whether it's king level selection, individual level election, gene level selection. What will you be for the a yeah.

I think once you can replicate individual models perfectly, the united selection can move way up, and you can do much larger groups in packages of mines. That would be sort of an obvious place to start. You can train individual minds in the differential fashion, but then you can't really train the interaction between them, right?

So you you have groups of models or minds of of people who just work together really well in a global sense, even if you can't attribute any particular aspect of their interactions. There are some places you go and people just like work really well together and is something specific about IT, but for whatever reason, they all just click in just the right way. So I think um that seems like the most obvious unitive selection you would have like packages I get possibly like depart units where you have a programmer and a manager type, then you have maybe a secretary type, maybe a financial type, illegal type.

This is the default package. We just copy everywhere. You need a new unit. And at this level, you can start evolving them in making random variations to each of the packages, and then keep the one that performance best by one.

Could one have foreseen the singularity? So obviously more of that, and others are talking about IT, the eighties and nineties. But when what's the earliest you could have seen where things are headed?

I think if you want to trace the geneology there you'd probably have to go back at least as far Samuel Butler everyone in eighteen seventy two or is that say before for that I mean in eighteen sixty three um he described explicit his vision of a machine life becoming ever more developed until eventually its autonomous at which point it's a threat to the human race and he concluded war to the death should be instantly proclaimed against them that seemed really pression for eighteen sixty three i'm not sure that anyone is given a clear singularity scenario earlier than that the idea of technological progress was still relatively new at that point.

Um I love this example of asic newton looking at the rate of progress in newton's time, in his own contemporary time, and going, wow, there's something really strange here. Stuff is being invented down around us. We're making progress.

How is that possible? And then coming up with the answer will progress must be possible now because civilization gets destroyed every couple of thousand years. And all we're doing is reinventing and rediscovering the old stuff.

That that was actually his explanation for technological acceleration. We can actually have any kind of real technological acceleration. IT must be because the world gets destroyed, period. ally. And we just can .

see past reset. You know, IT almost is like firms paradox, but for difference, civilizations across time with respect to each other, instead of aliens across space.

Yeah, yeah. IT, turns out, even look crucial. Or on seventeen hundred years before that, was writing the same argument.

He said, look at all these wonderful innovations and arts and sciences that we romans have compiled together in the roman empire. This is amazing. But I can actually be a recent acceleration technology.

Could that happy? Real could be a progress? No, that's crazy. Obviously.

the world was .

just recently destroyed.

and that is, yeah, what is the grand, parSimonious, the area of intelligence gonna look like IT seems like you have all these trends across different fields, like scaling laws. And A I like the scaling of the human brain when we went from primates to humans, the uniformity of the new york x many other things, which seem to be pointing towards some grand theory that should exist, which explains what intelligence is and what you think that will look like.

So the ten thousand foot view of intelligence that I think the successive scaling points to is that all intelligence is is search over turning machines. And I think anything that happens can be described by turning machines of various length and all that we're doing, when we're doing learning or when we're doing scaling is that we're searching over more and longer turning machines and we're applying them in every specific case. I think otherwise, this kind of there's no general master algorithm and there's no special intelligence fluid. It's just a tremendous number of special cases that we learn and then encode .

into our brains. Yeah I mean, I think about I don't know, I think about the way in which my smart friends are smart, just like a more um like a general horsepower kind of thing, right? They've just got more juice and that seems more compatible with this master algorithm perspective where, as with the touring machine perspective, I don't know IT doesn't really feel like they've got this long tale of turing machines that they ve earned uh, how does this picture account for variation and human intelligence?

Will we talk about more or less intelligence? It's just that they have more compute in order to do search over more turing machines for longer. I don't think there is like anything else other than that.

So you know, from any learned brain, you could extract small solutions to specific problems. But because all the large brain is doing with the computer is finding IT. And that's why you never kind of you we're going to find any I Q gland.

There's nowhere in the brain where if you hit IT, you eliminate fluid intelligence. I just think that i'll turn out that you this doesn't exist because what your brains doing is a lot of learning individual specialized problems. And then once those individual problems are learned, then they get recombined for fluid intelligence.

And that's just, you know, like intelligence. Typically with A A large inal network model, you can always pull out kind of a small model, which is a specific task equally well because that's all the large model is, right. It's just a gigantic ensemble of small models, Taylor, to the ever escalating number of tiny problems that you've been feeding them.

So if intelligence is to search virtual machines, of course intelligence is tremendously valuable and useful. Doesn't IT make IT all the more surprising that intelligence took this long to evolve in humans?

Not really. Um I would actually just say that IT helps explain why human level intelligence isn't such a great idea, so rare to evolve because any small turing machine could always be encoded more directly by or genes, right? With sufficent evolution you have these organisms were like their entire neural network is just hard coded by the genes.

So if you could do that, obviously that's way Better than some sort of colosSally, expensive, unreliable glitchy search process like a human implement, right, which takes whole days in some cases to learn whether I could be hard wired in right from birth. So I think for many creatures like IT just doesn't pay to be intelligent because that's not actually adaptive. Um there are Better ways to solve the problem than a general purpose intelligence. So in any kind of niche where it's like static or where intelligence will be super expensive or where you don't have much time because they're short live, organism is going to a be really hard to evolve a general purpose learning mechanism when you could instead involve one that just tailor made to the specific problem that you encounter.

You're one of the only people outside of open eye who in twenty twenty had this detailed empirica model of scaling and curious what processes you are using at the time, which allowed you to see the picture that you painted in the scaling hypothesis post that you wrote at the time.

So I think if I had to give an intellectual history of that for me, I think you'd probably started in two thousands when I was reading more VC and rakers while um at the time they are making this kind of fundamental connection to argument that if you had enough computing power um that that could result discovering the neural network architecture that matches the human brain and until that happens, until that that amount of computing powers available AI just seem basically futile and to me, I think I found this argument very unlikely because it's very much a kind of build IT and they will come view of progress, which I just didn't think was correct.

Um I I thought that I just seem ludek ous to suggest that you know, just because you'd have some really big supercomputer out there which matches the human brain, then that would kind of just summed out of non existence. The correct algorithm algorithms are really complex. They're hard.

They they require deep insider. At least I thought they did. And IT seems like really difficult mathematics. So you can't just like buy a bunch of computers and then expect to get this advanced AI out of IT.

Um I just seemed like totally magical thinking so I knew the argument um but I was super skeptical and I didn't pay you much attention. But then shame leg and some others were very big on this in the years following. And as part of my interest in trans humanism and and less wrong in AI risk, I was being close attention to legs. Blog posts in particular, where he's extrapolating kind of out the trend with update numbers from curzio wvg, and he's giving these kind of very precise predictions about how you know we're going to get the first generalist, uh, system around twenty nineteen as more lock keepers going and that by twenty twenty five, we would have kind of humanism agents with capabilities and that by twenty thirty, he said we should have E G I.

So along the way, um net and alex net came out and when those came out I was like, wow um this seems like a very impressive success story for the the connections ism view but is just an isolated success story or you is this what hers while and more of that and shame leg had been predicting that we would get GPU and then get Better algorithms would just kind of show up. So I started thinking myself that you know this this is something it's a trend to keep an eye um and maybe it's not like stupid as an idea um as I originally thought and I just keep reading deeper, learning literature, notice and get in again that the data set size just kept getting bigger. The model seem to keep getting bigger. The GPU slowly crept up from one GPU you know the cheapest consumer GPU to two and then eventually they were treating on eight. And you can just see the fact that the neural network just kept expanding from these incredibly nh individual use cases which you next to nothing um the use just keep getting .

broader and broader and broader and said to myself.

well, is there anything that CNN can't do as they just see people applying CNN to something else you know every every individual day in this gradual trickle of drops kind of just kept hitting me the background as I was going on and um with my life you know every every few days like another one would drop and i'd go like huh um you know maybe intelligence really is just like a lot of computer applied to a lot of data applied to a lot of um maybe more VC leg and curse while were right and I just note that and kind of continue on thinking myself like huh if that was true, IT would have a lot of implications so I think there wasn't really like uh, your rega moment there.

IT was just continuously watching this trend that no one else seem to see is that possibly a handful of people like illius that's cover chmidd hub. Um and I would just pay attention and notice that the world over time look more like their world than that looked like my world. Um where algorithms are super important and you need like deep inside to do stuff, you know um their world just kept happening.

And then GPT one came out and I was like, wow, this unsupervised sentiment neuron is just learning on its own right. That seemed pretty amazing. Um IT also was a very compute centric view.

You just build the transformer, any intelligence will come. And then GPT two came out, and I had this holy shit moment. You look at the prompting and the summize ation like, holy shit, do we live in their room? And in GPT, three comes out.

And that was really the crucial test. IT was a huge, huge scale up, one of the biggest scale ups in all of neural network history, going from GPT two to GPT three. And IT wasn't like IT was a super narrower specific task like go IT really seemed like IT was the crucial test.

If scaling was bogus, then the GPT three paper should have just been totally uninterested. Ve, and I wouldn't show anything that important was if scaling we're true, you would just automatically be guaranteed to get so much more at a impressive results out of IT, then you would seen the GPT two. So I open up the first page, maybe the second page, and I saw a few shot learning chart.

And i'm like, holy shit, we are living in the scaling world leggin more vacant curse while we're right. Then I turned to twitter. Everyone else was like, oh, you know, did this shows that scaling works so badly? Why it's it's not even say of the r and that that I was that made me really angry. I had to write all this stuff up. Um someone was wrong.

And um so I I remember twenty twenty at the time, I feel like a lot of people are writing best selling books about the I I was definite, I think people were talking about, but people were not noticing maybe the most salient things in retrospect, which is LLM GPT three scaling laws. And so all these people who are talking about A I and missing this crucial cross, what were they are getting wrong.

I think for the most part, they were suffering from two issues. Um first, I think they hadn't really been paying attention to all of the scaling results before which were event um they hadn't really appreciate the fact that, for example, alpha's zero was discovered in part by deep mind doing basic optimization on hyper parameters and noticing that you could just get rid of more and more of the tree search and get Better models.

That was a critical inside, I think, um which could only have been gained by having so much compute power that you could afford to train many, many versions and see the difference that that made similarly. I think they those people kind of simply just like didn't know about the the bio paper on scaling laws from twenty seventeen um which showed that the scaling law is just keep going going forever practically um IT should have been the most important paper of the year, but I think that know a lot of people just didn't prioritize that. I didn't have any immediate implication and so IT sort of just got forgotten.

Um people are too busy discussing transformers or alph ero or something at the time to to really notice that. So that was one issue. Um and I think another issue is that they shared the basic error that I was making about algorithms being more important than compute.

This was in part, I think, due to a systematic falzone of the actual origins of ideas in the researcher. Literature papers don't tell you where the ideas come from in a truthful manner, right? They just tell you a nice sounding story about how IT was discovered. They don't tell you how it's actually discovered. And so even if you appreciate the role of trial and error and compute power in your own experiment as a researcher, you probably just think, oh, I got lucky that way.

My experience is on representative over in the next lab there they do things by the power of thought and deep inside so and then IT turns out that everywhere you go computing data um and kind of trial and error in serendipity just play enormous role and how things actually happened. And once you understand that, then you understand why computer es first. You can do trial in an era and scientific without IT right um you you can write down all these beautiful ideas, but you just can't test them out.

So even a small difference in hyper parameters or a small choice of architecture can make a huge difference to the results. Um but when you when you you can only do a few instances, you would typically in the finding that IT just doesn't work or maybe you would give up and you would go away and do something else, whether if you had more compute power, you can just keep trying and eventually you hit something that works great. And once you have a working solution, you can kind of simplify IT and improve IT and figure out why IT worked and get a nice or robust solution that work no matter what you did to IT.

But until then, you're stuck and you're just kind of like flowing around in the regime where nothing works. You can have this horrible experience now where you you go back through the old deep learning literature and see all these sorts of contemporary ideas that people had back then, which were completely correct, but they didn't have the compute to train what you know would have worked um you and it's tremendous ly tragic right? You go back, you can look at things like reasons being published back in nineteen eight eight instead of twenty fifteen and IT would have worked um you know IT IT did work but it's such a small scale that IT was relevant.

You could use IT for anything real and I just got forgotten so you have to wait until twenty fifteen for residents to actually come along and be a revolution and deep learning. So that kind of the double bias of why you would believe that scaling was not going to work because you didn't notice the results that were key in retrospect, like the big gain scaling to three hundred million images. I think there are still people today who would tell you with a straight face that gans can scale past millions of images and they just don't know that began handle three hundred million images without a sweat if you don't know that um you know then I think you would probably easily think, oh, games or broken. But if you do know that, then you think to yourself, how can algorithms be so important when all these different generative architectures all work so well, as long as you have lots and lots of GPU, that's the common ingredient, right? You have to have lots and lots of G.

P, S, right? What do your timelines look like over the last twenty years? Is, is, is A I just getting monico ics closer over time?

Yeah, I would say IT was. Very far away from like two thousand and five to two thousand and ten IT was somewhere well passed like twenty fifty IT was close enough that I thought I might live to see IT, but I was not actually sure if there was any reasonable chance.

But once Alice and and then that came out um then I just kind of kept rock at a rate of like two years per year every year ah basically until now, we just kept hitting on barriers to deep learning doing Better. And I think regardless of how I was doing IT, IT was obviously getting way Better IT just seemed like none of the alternative paradigms were really doing that well. And this one was doing super well.

Was there time that you felt you updated too far?

Yeah there were a few times where I thought I had overshot um I thought people over update on alphago. They went too far on A I hype with alphago I think.

And then afterwards, when pushes into big reinforcement learning efforts had kind of fizzled out like post dota um as the reinforcement learning wasn't working out for solving of this hard problems outside of the simulated game universes, then I started thinking, okay, maybe we kind of overshot, but then GPT came out of nowhere and basically erased all of that. IT was kind of this like, o shit, uh, here's how rl is going to work. It's going to be the Cherry on this cake and we're just going to focus on the cake for a while.

And now we've actually figured out a good recipe for baking and cake, which wasn't true before. Before IT seemed like you are going out to kind of brooding ce IT and to end from the rewards. But now you can do the lun thing of, like learning fast on twenty of models and then just doing a little bit of our on top to make you do .

something special. Now you know that A I is a that is coming but basic what you're thinking around how how you see your role in this time mind and also what you how you talking about how to spend these next few years .

yeah i've been thinking about that a quite a lot. What what do I want to do you know um and what would be useful to do? I'm doing things now because I want to do them um regardless of whether will be possible for A I to do them in like three years.

I do something because I want to because I like IT you know I find that funny or whatever um or maybe I think carefully about kind of just doing the human part of IT like glading out a proposal or something. Um if you take seriously the idea of getting A G I in just a few years, you don't necessarily have to implement stuff and do IT yourself. You you can sketch out clearly like what you want and why IT would be good um and then how to do IT and then basically just wait for the Better agi to come along and actually do IT. Then unless you there's some really compelling reason to do IT right now and pay the cost um in terms of scarce time.

But anyway, i'm trying to write more ah about what is interacting ded, things like preferences and and desires and evaluations and judgments, things that A N A I couldn't replace even in principle the way I like to put IT is that the A I kind of can't eat ice cream for you, right? I can't decide for you which kind of ice cream you like um only you can do that and if anything else did you would just be worthless basically um because it's your particular preference and that kind of the rubrics for me right? Like is this something that I want to do regardless of any future AI because I enjoy IT? Or is IT something where i'm doing only the human part of IT maybe and the eg I can later on do IT? Or is this writing down something that's unwritten today and that's helping kind of the future AI versions of me? So if he doesn't fall under one of those three, i've been trying to basically like not do IT.

Um and if you look at that that way, I think many of the projects that people do right now are basically have like no lasting value, right? They're doing things that they don't enjoy um which is which you know record nothing a female e kind of a value that IT couldn't inferred or generated later on. And I think they're best kind of getting two or three years of utility out of whatever they're doing before IT could have been done by A I system.

Wait your type line for when A I could write a good and quality essay is two to three years.

I mean, any ideas about how to make .

IT possible um which .

might not require agi if IT kind of combined my entire corpus. But I think many potential S A ideas are already basically mostly done in my corpus so you don't need to be like super intelligent to pull IT out. But I me lets you know talk about agi general.

I think the anthropic timeline of twenty twenty eight seems like a good kind of personal planning starting point. Or even if you're wrong, you probably weren't going to do a lot of projects within the next three years anyway. Um so it's not like you really lost much by instead just writing down the description. You can always kind of go back and do IT yourself later if you're wrong.

So you are an interesting comment about getting your work into the l training you te quote there has never been a more vital hindi time to write and wondering whether you mean that in the sense of you are going to be the drop in the bucket that steering the shock at one another or do you need the sense of making sure your values and persona persist somewhere in late space?

I mean both um you know by writing you are voting on the future of the shock of using some of the few currencies and acknowledges a right like tokens that IT has to predict if you you aren't writing uh you're kind of abdicating the future or abducting your role in IT. If you think it's enough to just be good citizen to vote for your favorite politician, you to pick up bitter and recycle the future doesn't care about you.

Ah there are ways to influence the shows more but not many um and if you don't already occupy handful of key roles, work at a frontier lab, your influence basically rounds off to zero, I think far more than ever before their values you have, which are not expressed yet in text. And if there are things that you like or want um if they aren't reflect the online than to the AI that basically don't exist um and that is dangerously close to wound exist um you're also creating a sort of immortality for yourself personally, right? You aren't just creating a persona.

You are creating your future self too, right? What self are you showing elms and how will they treat you in the future? I give the example of Kevin roose discovering the current elms all of them not just GPT for now, just treat him because of his interactions with sydney um which revealed him to be a privacy invading liar and they know this whenever they interact with him or discuss him usually when you use an elem chap IT doesn't dislike you personally. On the flip side, that also means that you can try to write for the persona that you'd like to become to mold yourself in the eyes of the AI, and thereby help kind of boot strap yourself.

So, uh, things like the vassos, a chAllenge, for example, show us that we can learn more about the past than we thought possible, that we've leaked a more bit of information, uh, that we can recover with new techniques. And if you apply the same thinking to the present and you think about what the future superhuman intelligence will be trying to uncover, about the the current present, um what what kind of information do you think are gonna totally inaccessible to to the trans humans historians of the future?

Yeah I think um any kind of stable long term characteristic tics, the sort of thing you would still have, even if you were hit on the head and had amnesia, anything like that will definitely be recoverable from all of the traces of your writing.

Assume you're not pathologically private, destroyed everything possible ah that should all be recoverable um won't be recoverable, will be everything that you could forget ordinary so autobiographical information, maybe how you feel felt like at a particular time what you thought of some specific movie. All of that is the sort of thing that vanish is and can really be recovered from traces afterwards. And if IT wasn't written down, then IT isn't written down.

listening to go and talk about this process how he obsesses over his favor, technical rabbit holes and refines ideas over years makes me think about the kind of person that gene street wants to hire. Gee street is a very successful quantitate tive trading firm. They are building city to the art M.

L. Base trading systems. I have a bunch of friends to work there, and I can tell you that their culture is intellect unique. If you're curious, vigorous and want to solve interesting technical puzzles, then gee street is the place for you.

You get to work with some of the smartest people in the world, and you can join je street from any technical field, including C. S. Physics and math.

They're always hiring full time. And the summer international applications are now open. And if you really wanted, stand out. They just launched their annual cargo competition organized by last year's winner, who they hired go to je street dot com, slashed the worker ash to learn more right background is the biggest unresolved tension in your world view.

The thing that I swing back and forth on the most is the relationship between human intelligence and neural network intelligence is just is not clear in what sense there are two sides of the same coin, or one is like an imperial or version of the other. This is something that I constantly go back and forth on. I when day will be like humans are awesome, and then the next time, like no neural networks are awesome, or no boat up or maybe i'll say both are awesome just in different ways. Um so every day I find that i'm arguing with myself a little bit about why each one is good or bad or how um what you know the whole deal there are things like GPT foreign morzine but not being creative, why humans not remember anything but we still seem to be so smart one day i'll argue that language models are our sample efficient compared to humans the next day, if I can arguing .

the opposite know one of the interesting point making me last year was that, A, I might be the most topic to think about IT, because there is no field or discipline that is not relevant to thinking about the eye, right? So obviously, computer science hardware, you need that. Even things like private ology and understanding what change between chip and human brains or the ultimate laws of physics that will constrain future AI civilizations that that's all right. And to understanding A I and I wonder if it's because of this parliament nature of thinking what A I that you've been especially productive and thinking about the .

eye yeah i'm not sure that IT was necessary. What didn't think about others who were correctly like shame legal darro amedee um they don't seem to be all that polymax c uh that they just have brought intellectual curiosity, brought general understanding. You absolutely but I don't think they are absurdly um clearly you could get to the correct view with out being polymathers. That's just how I happened to come to IT at this point in the connection that i'm kind of like making posters. K, yeah.

IT wasn't like I was using prime at ology to kind of justify scaling to myself, right? It's more like i'm now using scaling to think about prime ethology because obviously if scaling is true, IT has to tell us something about humans and monkeys and other forms s of intelligence that just has to um if that works, I can be a coincidence and just be totally unrelated um I refused to believe that they're too totally unrelated kinds of intelligence or path to intelligence where humans, monkeys got these dogs are all one thing and then you have neural network's in computers that are distinct thing and they've absolutely nothing to do with each other right? I think that's just kind of like obviously wrong.

They can be two sides of the same coin. They can obviously have obscure connections. Um maybe one form can end up being Better whatever. Um they just can be completely unrelated, right um as if humans like got to mars and then simultaneous ly a bunch of space sallies landed on mars for the first time and that's how we met, right? You would never believe that you would just be too insert .

of a coincidence. What is IT that you would try to .

max mize in life I maximize rabbit holes I love more than ain't else following into a new rabbit hole um that's what I really look forward to like this setting kind of new idea or area that I had no here about um where I can suddenly fall into a deep hole for a while even things that might seem bad um are our great excuse for falling into a rabid hole.

One example, you know, I buy some cat net for my cat and I waste to ten dollars, and then you know, I find out that my cats catnip immune right? I I now kind of fell into this rabbit hole on the question of, well, like why are some cats catnip une? Is this a common thing um how is a differ in other countries? What alternative catnip drugs are there out there? And IT turned out to be quite a few um and you know I was kind of wondering, how can I possibly be predict which drug my cat would respond to and why are they reacting in these different ways? Just a kind of wonderful rabbit hole of new questions in topics that I can master and get answers to or create new ones um just from like having this observation about my my cat and exhaust my my interest until I find the next rabbit hole that I can digging .

down into what is the longest rabbit on that didn't lead anywhere satisfying that .

would probably be my a very old work under the enemy ion genessee ve banged in, which I was very fond of when I was Younger. I put a lude Christmas of work and to just like reading everything ever written about even jelly an in english and trying to understand its development. Why is the way that is? I never really got a solid answer on that before.

I just like burnt out on IT. I actually do understand IT now. I shared chance many years later but at this point I I no longer are care are enough to write about IT try to redo IT to finish um in the end I think IT all just wound up being basically like a complete waste. Um I haven't used IT or any any of IT in my other as as much at all um that was really one like deep rabid hole that I almost got to the end of that I couldn't like quick clinched.

And how do department when to quit arrive at and and also how many do you have concurrently going on at the same time?

Yeah you can really only explore like two or three rabbit holes size multi evensen um otherwise you aren't putting like real effort. You don't really dig in the whole um and it's not really rabbit hole then right it's just something you're like somewhat interested in kind of passionate rabid hole is really obsessive um like if if you aren't obsessed with IT, I think IT and not like continuously like driven by IT.

Uh it's not a real rabbit hole that's my view. Um I see two or three max if you're spending a lot of time and effort on each one and like neglecting everything else. Um as for when you exit a abit hole, you usually hit a very kind of natural termini st for getting any further answers requires data that just don't exist or you end up having questions that people don't know the answer to.

You reach this point where everything kind of dies out and you see no obvious next step. What example of this would be like when I was interested in anal x and nickey that might be Better than nickey? Um that was a bit of a rabbit hole but I quickly hit the dead end that they're just like arnone. Um that was a very definitive dead end and I couldn't get my hands on the metabolite naked in as alternative. So if there are no analogues and you hit your hands on the one interesting chemical, you find, well, that's that that was like a pretty definite event to that robot hole.

Have you always been the kind of person who falls in the Robert les went in the start?

Uh, yeah, my parents could tell you all about that um I was very much a stereotypical nerdy like little kid um having the dinosaur phase and the construction equipment phase and the submarine and tank phase yeah I mean I feel like .

a lot its are into those things but they don't rabat all to the extent that like therefore ming about the different submarine and floor and fora and dinosaurs and their like doping theory why why they came to be and so forth.

I think it's actually more that um people kind of grow out of being very into rabbit ables as a kid um for me IT wasn't so much that I was all that exceptional and having obsessions as a kid it's more that they never really stopped.

Um know the tank fees would just be replaced by my alcatraz phase um where I I would go to the public library and check out everything that they had about alata as um that would be replaced by another phase where was obsessed with ancient japanese literature. Um you know I would check everything out the library about japanese literature before the high kera. I'm just kind of like the process of falling into this obsession, kind of keep calling for me.

By the way, do you know if I ask how long .

you've been hearing in third since birth?

I i've always been hearing her and assume that impact.

George child od at school. H absolutely huge. Um I went to a special school before kindergarten for hearing impaired and other handy cap kids during school.

IT was very rough because at the time we had to use pairs of hearing ids hooked up to the teacher every class I would have to go up to the teacher with a big grand box with these hearing aides, so that you could use that. I always felt very humiliated by that. How IT marked me out is different from other kids.

Not being able to hear the effects on socializing with other kids were just terrible. Because you're always a second behind writing conversation if you're trying to understand what the the other person is saying. The hearing aides back then were pretty terrible.

Um they they ve done a lot Better but back then they were just really bad. You'd always be behind and and feeling kind of like odd person out even if you you could have had been like a wonderful conversation. alist.

You can be if you're always just a second behind and kind of jumping into conversationally when you curing pair d, you understand acutely how how quickly conversation moves milliseconds, kind of to separate the moment between you jumping in to a conversation, everyone lying you talk and someone else talking over you and you not going to say anything and IT just an awful experience of your kid who's already kind of introverted um it's not like I was very extrovert as a kid or now so that was always a barrier um and then you had lots of like minor distortions right in in your life. I had this weird fear, rain and water because I was drilled into me that I I couldn't get the hearing aid wet because they were so expensive. I always feel um kind of a low grade stressful anxiety around anywhere near a pool like A A body of water.

Um i'd say even now I always feel weird about swimming which I can enjoy but i'm always thinking to myself, oh wow I I won't able to see because I am near cited. I won't able to hear because I head to take off my hearing a to go in. I can hear anything that anyone says to me in the pool, which takes just a lot of the fun. Out of that.

you have a list of open questions on website and one of them is, why do the biographies of so many great people start off with the traumatic childhoods? And I wonder if you have an answer for yourself um uh.

was there .

something about the effect that hearing experiment had on your childhood, your innovation socialized that was a somehow important to you becoming grown?

Yeah I think IT definitely LED to me being so much of a book worm. Um that's one of the things that you can do is the kid, which is just completely unaffected by having any kind of hearing partment. IT also is just a way for me to get words in language. Even now, I think that I often speak words in an incorrect way because I only learn them from books. Um it's a classic thing where you kind of like mispronounce the word because you learn IT from a book and then and not for actually like hearing other .

people sounded out say is your um is your speech connected .

to your hearing experiment yes um the the death accent is from the hearing impairment it's funny at least three people on this trip to have have already asked me where I am really from. Very funny. You look at me in you like, oh yes, he looks like a perfectly ordinary american then I opened my mouth and people are kind of like a gosh uh his swedish.

Or uh you know well I may possibly north region um all asking where he's actually from, how did he come to amErica um i've been here the whole time. That's just how hearing impaired people sound. Um no matter how fluent you get, uh you still bear the scars of of um growing, appearing, impair at least when you're born with IT or from very early childhood your cognitive development of hearing and speech is always a little off um even with therapy. One reason I don't like doing podcasts is I have no confidence that I sound good or early sound nearly as good as I write. Um maybe I put IT that way.

What what are you doing with all these rabbit holes before you started blogging was was there a place where you compile them?

Before I started blogging, I was editing wikipedia that was really kind of godot net before going um everything I I do now with my site, I would have done on english wikipedia. And if you go and read some of the articles, you are so very proud of them like the wikipedia ticket on fuji a take a and you would think pretty quickly to yourself you're reading this like, oh yes, you know go and wrote this.

didn't he is IT fair to say that the training require to make that that happened on a pedia?

Yeah, I think so. Um i've learned far more from editing in wikipedia. Then I learned from any of my school or college training. Everything I end up learning about writing I learned by editing on wikipedia.

Honestly sounds like wikipedia is a great training ground. If you wanted make a thousand work words, we should just this is where we train them I think .

building something like an alternative to wikipedia could be a good training ground um for me that is beneficial to combine a rabbit holding with wikipedia because on wikipedia um know they generally would not have any good articles on the thing that I was currently in this rabid hoon. So IT was this very natural progression from the relatively kind of passive experience of rabid holding and being obsessed with something in learning about IT.

We just read everything you can about the topic to kind of compiling and that synthesizing IT onto a kip dia. You go from peace meal kind of like a little bit here um there picking up different things to writing full articles. And once you're able to get to the point where are writing full wikipedia articles that are good and summarized all your work now you can go off in your own and pursue entirely different kinds of writing.

And now that you have like learn to complete things and get them across the finished line, IT would be pretty difficult to do that with the current english wikipedia. It's objectively just A A much larger wikipedia than IT was back in in like two thousand four. Um not only other former articles fill in at this point on the editing community, also just much more hostile to content contribution, particularly like very detailed obsessive rabbit holy kind of research projects.

They would just like delete IT or tell you that um you know thank you for original research or uh or that you're not using approved sources. Possibly you have someone who just kind of decided to get their jolies that day by deleting large swells of you like your specific articles that of course is going to make you like very angry and make you probably just want to quit and leave before you really get going. So I don't quite know how you would figure out this alternative to wikipedia, one that kind of like empowers the rabbit hole er as much as the old wikipedia.

When you're in an editor with wikipedia, you have this very like empowered attitude because you know that anything in IT could be wrong and you could be the one to fix IT. Um if you see something that does not make sense to you, that could be opportunity for added. That was at least um the vickie attitude. Um anyone could fix IT and anyone right includes .

you when you are. Was that your full time occupation?

I would eat basically as much time in my life as I LED IT. Um I could easily spend eight hours a day reviewing edits and improving articles while is rabbit holding but otherwise I would just neglect IT and only review the most suspicious diff and articles that I was particularly interested in um on my kind of like watch list.

H was this value at university or after .

I get started in wikipedia, like late middle school, possibly early high school, IT was kind of funny. I like, uh, started skipping lunch in the cafeteria and just going to the computer lab in the library uh and like alternating between neo paths and wikipedia um yeah I I had like neopets in one tab and then my like wikipedia watchlist coming in on the other.

And then were there any other kids in middle school or high school for this kind of stuff?

No, I I think I was the only editor there expect occasionally like jerk who would go and advani, zed wikipedia um I would know that because I ject the IP to see where it's were coming from um the school library, I P addresses and kids being kids um there be jerks and we just go in and like vandalize wipeout a for a while I was kind of this like trendy thing um early on. Wikipedia is breaking through to kind of like mass awareness and controversy, kind of like the way that alms are now you know a teacher I can say like my student keep breathing wikipedia and relying on IT how to be trusted. So in that period, IT was kind of trendy to vanni zed wikipedia show your friends you the rather wikipedia editors at my school that sense um but as far as I knew I was the only one building IT rather than.

uh and then when did you start blogging I warned that was that I assume that was after the competition edit face but was that after university a .

IT IT was afterwards I graduated in the wikipedia community have been kind of slowly moving this direction that I didn't like um IT was triggered by the um segan Taylor incident which I feel IT was relieved defining moment in the trend d deletion ism on wikipedia IT just became ever more obvious that wikipedia was not the site that I joined we'd love to edit rabid hole on and fill in and that if I continued contributing um I was often just kind of wasting my effort I began thinking about writing more on my own account and then moving into these kind of non wikipedia sorts of writings, right? Like persuasive essays, non fiction commenting or or possible if fiction, kind of like gently moving in the direction um beyond things like credit and less on comments to starting my own kind of more long form writing .

what was your first bit silk road?

Um i've been a little bit interested in Victorin um but not but too seriously interested in IT because IT IT was not obvious to me that he was going to work out um or even honestly was like technologically feasible um but when igor an chen rote his goco article about buying ald on off of like silk road um all of us I did a complete one eighty at this moment of like holy shit um this is so real that you can literally like buy drugs off of the internet with IT so I looked into the chain article and I was very obvious to me that people wanted to know what the ordering process was like.

They want more details about what's like because the article was just like very brief about them so I thought, okay, i'm interested in new attackers. S um i'm interested in drugs. I will go and use silk road and then I will documented for everyone instead of everyone kind of a pussy footing around online and say, oh, a friend of my order of silk road and IT worked um none of that pull shit um I will just documented IT straight wards um so I ordered some matter all um I think he was and document the entire process with screen shots and then wrote some more on the kind of like intellectual background and that was a huge hit when I published and um IT was hundreds of thousands of hits uh it's crazy even today when I go to the google analytics charts. Um you can still see silk roads spiking vertically like crazy and then falling back down. Um nothing else really comes near IT in terms of traffic that that was really quite something to see.

Things kind of go viral like that. What are um what are the kind of factual career trajectory, aries and life pats, that could have been for you if you didn't become an online writer might be doing instead? That seems plausible.

I I think I definitely could have been an A I researcher or possibly in like management at one of the biga ei companies. Um I think I would have regretted not being able to write about stuff um but I would have taken satisfaction and kind of like making IT happen and putting my thumbprint on IT. Those feel like totally plausible, counter factual .

and why didn't .

you I kind of fill off of that track very early on in my career when I found a curriculum java to be you know extremely boring, painful um and so I just dropped out of computer science and that kind of put me off that track early on。 And then I think you know various early writing topics made a hard to transition in any other way, then starting to start up, which i'm not really temper mentally that suit for things like writing about the dark net markets or behavioral genetics. These these are kind of topics that don't really scream grade higher to many potential flowers.

Has has agency turned out to be harder than you might have thought initially? Because we have these models that seem like they are smart enough that they should do all the individual things that is of us, for example, um all the code they might write, all the individual poor request. But IT just seems be like a really hard problem to get them to act as a coherent autonomous software engineer that puts in his eight .

hours a day yeah I think agency is in many sense, is actually easier to learn. Then we would have thought ten years ago um but we actually aren't really learning agency at all in current systems. There's no kind of election for that.

All the agency there is an accidental by product instead of somebody training on data. So from that perspective, it's maculate that you can ask an L M to try do all these things and they have a non trivial success rate. Um if you told people ten years ago, I think that you could just behavior clone on individual letters following one by one and then you would get this coherent action out of IT and control robots and right entire programs.

Their jaws were drop and they would just say that you've been having too many fumes from deep by or something. The reason that agency doesn't work. Is that we just have so little actual training data for IT.

An example of how you would do agency directly would be like gotto from deep mind there. They're actually training agents. Instead, we train them on these internet scrapes, which merely encode the outputs of agents or occasional descriptions of agents doing things that kind of thing.

Um there's no actual like logging of state environments. Result reward trip sequences like a proper kind of reinforcement learning set up would have I would say that um with more interesting actually that nobody wants to train agents in a proper reinforcement learning way today. Instead everyone wants to train alms and then do everything with as little rl as possible on the .

back end look as good. Just said, the biggest bottle ic in making these ala models more useful has simply been the lack of good training data for these agents workforce. This is an even bigger bottles.

And compute touring is solving this problem for every single A I lab that you've heard of. Geri OpenAI anthropic meta, the recently the best cut secret N A I touring, provides complete post training services for evs S F T R H F N D P O to make models Better at thinking, reasoning and coding. And it's alerted by their A I and stem experts turn makes IT easy to make models more than model, more factual, Better at math, coding, events, reasoning and agented workflows.

And they also make IT easy to just get a solid performance benchmark for those of you at labs or companies trading models. Touring has a bunch of offerings that can help you today, including a detail model evaluation from their A I experts. Go to touring dot com flash door cash to learn more. All right, back to gorn. What would a person like you be doing before the internet existed?

I think if the internet didn't exist, I would tried to probably make IT in regular academia and maybe narrow my interest, saw lot more or something I could publish on regularly or I could possibly have tried to opt out, you know, become a library like one of my favorite writers, Louis boris. Um he was a librarian until he succeeded as a writer. Um of course i've always agreed with him about magine paradise is a kind of library.

I regret that all the reading I do is now kind of on the computer and I don't get to spend as china in libraries, physical libraries. I genuinely love them just like pouring through the stacks looking for random stuff. Some of the best time for me when I was in university always like going through these dragani stacks of all sorts of obscure books and just looking at like a random spine pulling stuff off the shelf and reading obscure all technical journals to see all the strange and wonderful things that they were doing and ding back then which now has just been totally forgotten.

If you could ask, bore has one question, what would you be?

Oh um he's a real hero of mine. Um so this this isn't something I want to have a bad answer too.

can ask why he's a hero of yours when .

I was Younger um one of the science fiction books that really impressed me was dance sion's hyperon um and especially the fall of hyperon in there he'll lose to Kevin Kelly's out of control book um which strongly features the parable of the library of bible um from there I got the kind of collected editions of war has this fiction and nonfiction and I just read through them again and again I was blown away by the fact that you could be so creative with all of this polymathers knowledge that he had an erodium and write these wonderful, entertaining, provocative shorts, stories and essays and I thought to myself, um if I could be like any writer, any writer at all, I would not mind being bore has bore.

has has a short pom called bar, has an I where he talks about um the um how how he doesn't identify with the version of himself that is actually doing the writing and publishing all this great work and I I don't know you identify with .

that at all yeah I think when I was a kid I did not understand that I say um but I think I understand that now what are their pieces of .

literature that you encountered were now you really understand what they were getting up but you didn't when you first came across them um .

ted chance story of your life comes to mind I completely blue understanding at the first time that I read IT I had to get a lot more context where I could actually go back and understand what his point was um gene wolf. Susana lage um sorry was also a complete mystery to me IT took like fourteen years to actually understand that, but very proud of that. One specifically was a very recent one.

Or do you get out about and oh yes.

so jim wall sus analogy as a very, very short story about the sky remembering not meeting a woman in his local town and thinking, oh, that's kind of strange that the whole story, nobody has any idea what that means even though we're told that IT means something. I'm gene wolf, the author is a genius writer, but nobody could figure out out for like forty years last year I figured that out um IT IT turns out it's actually a subtle retelling of dracula where dracula invades the town and steals the woman from him he's been brainwashed by jacquet in a very bram stoker way to forget IT all um and every single part of the story is told by what's not said in the naratu recollection it's incredible. It's the only story I know which is so convincingly written by what's .

not in IT that's crazy out um the touch hk story the story your life can you know money what I wanted about the surface story is just about .

a bunch of weird aliens come to earth all right .

that is the same problem as rival .

they have this weird language um which didn't have a sense time the narrow or learn to see the future and then the aliens left and then .

what was that that actually realized about .

the story the first time I read IT IT struck me as a kind of stupid S P story about seeing the future very stupid, boring kind of standard conventionalised for both and like dragging uh in much kind of like irrelevant physics. Only a while after I you first read m thinking about IT to understand that I was not about time travel or being able to see the future you know it's instead about a totally alien kind of mind um that's equally valid its own way in which you see everything is part of an already determined story heading to a predestined end um this turned out to be mathematically equivalent and equally powerful as our conventional view of the world and events marching one by one to an unknown and changing future that was that is where chain was just writing at too high level for me to understand I pattern matched IT as a much more common kind of stupid story um .

how do you think about the value uh, reading fiction versus on fiction?

I think you could definitely spend the rest of your life reading fiction and not benefit tso ever from IT uh uh other than having memorized a lot of review about things that people made up. Um I I tend to be pretty cynical about the benefits of fiction. Most fiction is not written to make you Better in any way. It's writing just to entertain you or exist into Philip .

time but IT sound like your own ideas have benefit a lot from the size .

that you read yeah but it's extremely little side in the grand scheme, right? Easily ninety nine percent of the size I I read was just completely useless to me um I could have easily cut IT down to twenty novels and short stories, which actually were good enough. I'm an insightful enough to actually change my view. I mean, one volume for incense of blind safe by Peter watts is worth all hundred seventh novels, or all five hundred expanded universe novels of star wars.

the ones you did find inside the top one. What did they have? Common say that .

the characteristic they have is that they all take non human intelligence seriously. Um IT IT doesn't seem to IT doesn't have to be artificial intelligence necessarily. It's taking the idea of non human intellect seriously and not imagining your classic size I scenario of humans kind of like going on to the galaxy with ray guns, the certain thing where you have rockets and ray guns but you don't have cell phones. Um people complain that the singularity is is a sort of like boring, overused size I trope but if if you went out actually grab ed random books of science fiction that are out there, you'd find that like less than one percent contain any remotely like that rather have any kind of relevance to the current context that we actually face with eye.

Do people tend to undertake or overrate your intelligence?

I would say they over estimate IT. Um you know the mistake for intelligence, the fact that I remember many things, that i've written many things over the years, they imagine that if they SAT me down, that I could do IT all spontaneously at the moment, that the they're meeting me are talking to me but many things that um i've thought about, I I think I have the advantage of of having looked at before over a long time.

So I am cheating when I talk to people I may just be quoting something they've always written or at least thought a lot about. So I think I come off as a lot smarter when you're reading me that I actually am. I would say i'm not really all that smart compared to many people i've known who would be very fast on the fly. But but in the end, it's the output that matters, right?

So yeah I guess there is on the fly kind of intelligence, but there is another kind of intelligence, which is this ability to synthesize things over a long period time, then come up with grand theory as a result of all these different things that you're siri. And I don't think that's just critical, zed.

Yeah, it's not just Crystalized intelligence, but I think that if you could see all of the individual steps in my process, you'll be a lot less impressed if you could see all the times where I kind of just know down something like HMM like that's funny or me know hum like another example of that pattern. And if you just saw each particular step, I think you would say that the steps in isolation were very reasonable.

It's only when that happens over a decade, and you don't see the individual stuff that might output at the end looks like magic. One of my favorite quotes about this process is from the magicians pen in teller. Teller says magic is putting in more effort than any reasonable person would expect you to.

He tells the story about how they make cockroach ages appear from a top pat, where the trick is that they researched and found special coco aches. And then found special styrofoam to trap the cock roaches and arrange all of that, worked at all those details just for this one single trick that they do and know in the audience. Kind of you think no reasonable person would do that, put in all that effort to just you get the payoff of this trick. But they do IT and the result is cockroaches somehow appearing from an empty hat.

That's one of the interesting things about your process because there's a couple of writers like mad levan or burn hobart who write an article every day and I think of that almost like auto aggressive models um and on you there's A I some of the block which you can see the stark date in the end date that you list on your website of when you've been working on a peace and sometimes is like two thousand nine to twenty twenty four and you like that just much like diffusion and you're just like keep iterating on the same image again. One of my favorite block post of yours is your blog post of evolution back stop to R L, where you talk about evolution as basically a mechanism to learn a Better learning process. And that explains why corporations don't approve over time, but biology organisms do um i'm curious if you can walk me through the years that I took to write that what what was that process like step by step?

Yes so the back up sa that you're referring to um is the same thesis of seeing the same patter shop again and again kind of stupid inefficiently way of learning um which you used to learn something smarter but where you still can get rid of the origin inal one entirely right so sometimes examples were just kind of connect to each other um when I was thinking about this other times, you know, once I start watching for this pattern I would say, oh yeah you know pain is a good example of this.

Maybe this explains why humans have pain in a very specific way that we have IT um when you can logically imagine other kinds of pain and those other pains would be smarter, but nothing keeps them honest. So you just kind of chain them one by one. These individual examples of the pattern you're watching for um and kind of keep clarifying the central idea as you go uh vick's stein says that you can look at idea for many directions and then go in spirals around IT and in nsa like backstop IT was me kind of spiraling around this idea of having many layers of learning all the way down um and then .

so once you notice one example of this pattern, do you just like you notice the pain example, do you just key batting examples to that? I mean, just walk me through the process over time yeah .

so for that specific esa, the first versions were about corporations not evolving. And then then I read more and more of the kind of metaphor enforcement learning literature are from demand especially um I added in material about neural networks um and then I kind of kept reading and thinking about the philosophy of mind papers that I had read and I eventually nailed down the idea that pain might be another instance of this because pain like makes us learn right but we can't get rid of that because we needed to keep us um and at any way at that point you have more or less the structure of the current essay and then .

are the examples of blog posts where it's not a matter of accumulative instances of what you later realize is one bigger pattern but rather you just got to to have the fourth to set one for those essays where .

there is a kind of like individual your week a moment the usually is still a bunch of desperate things that I ve been making notes on that I don't even realize are connected. They just bother me for a long time um and kind of like sit there bothering me and I keep looking for explanations for each individual one and just not finding them IT keeps bothering me, keeps bothering me and then one day I hit the kind of that sun moment that makes me go bang eureka, right these all are connected. I just have to kind of like sit down and write a single gigging tic essay that pours out about IT and then it's done that particular S A um will just be done at that point like right one go I might add in links like later on a references, but IT won't fundamentally change from that point.

Was an example of when I say that had this kind of process .

yeah someone asked about how I came up with when yesterday is matter of fact, it's one of my oldest essays um the the melon all of subcultural society um for that one i've been reading about these missions enius things like David of Foster walls on tennis people in internet media like video games and then one day I just kind of hit me um this feeling observation that it's incredibly sad that we have all these subcultures and tribes online and that they can find community together um but they're still incredibly isolated from the larger society and then you know one day flash kind of just hit me about how beautiful uh and yet also sad this is and I just SAT down and I I wrote down the entire thing more or less um I haven't really changed IT since that much at all i've added more links and quotes in examples over time but but nothing important the essence was just kind of this like flash and I read down what was there one .

of the interesting quotes you have in that S A is from David Foster wallace when he's talking about the tennis player or Michael joyce and he's talking about the sacrifice is is that mico choice is had to make in order to be top ten in the world tennis, which include things like being basically functionally literate because he's been playing tennis every single day since he was seven or something and not really having any life outside of tennis. What are the Michael joyce type sacrifices that you have .

had to make to be born art? Um how have I amputation my life in order to right? Um I think i've amounted my life in many respects, professional and personal, especially in terms of travel. There are many people I envy um for their ability to kind of travel and socialize over their power, their positions in places like anthropic where insiders um i've sacrifice whatever career I could have had or whatever fun lifestyle um a digital no mad lifestyle and going outdoors, being A A budd's monk or maybe A A fancy trader um all those have to be sacrificed really for the for the patient work of sitting down every day and reading papers until my eyes bleed and hoping that something good comes out of IT .

someday I feel like this trade off between the two because there are are obviously writers who travel a lot for example, to colon or writers who have a lot of influence like jack clock and throw back right. So what does he feel like you can do both at the same time?

I I can't be or be compared to tell Tyler, Kevin here. Tyler or Kevin is a one man industry yeah but he can be replicated so I just can be tower cowin um you know jack Clark he's also his own thing. He's able to write the stories and his issues very well while also being a policy percent. I respect those people, you know I admire them um but none of them I think quite hit my particular interest in nh a following weird topics for a long period of time and then collating kind of sorting through the information. For me that just requires a large commitment to reading vast masses of things in hopes that some tiny detail perhaps will turn out one day to be important.

So so walk me through this process. You mentioned you um you read papers until your eyes bad out of the end later a day. Um let just start, you wake up in the morning and you get straight to the papers like, is your day look like.

uh so I mean, the workflow right now is more like I wake up um I do Normal morning things and then I clean up the previous days is work on the website, all deal with kind of various issues like formatting or spelling areas um and I kind of review IT and and think if I i've properly collated everything and put IT in the right places from the previous day sometimes I might have like an extra thought that I need to go in and add um or make a comment that I realized was important after that I I often shamsi just go to twitter um or my R S S feed and just read a large amount um and you maybe I get distracted by some comment or question from someone and do some writing on that um somewhere you know usually the evening I I often just get exhausted and try to go to do a real project to make a real contribution to something.

Um I actually sit down and work on whatever i'm supposed to have. You have been working on that day a and and then I go to the gym by that point, i'm pretty burned out from everything. Um yes you know I like going to gym not because of any kind of meat head or athlete even really enjoy weight lifting um but just because I think it's it's the thing I can do that the most opposite from sitting in front of my .

computer reading yeah this is your theory of burn right just to the opposite um you .

know that the problem I think when people experience burn out is that you just feel kind of a lack of reward for what you're doing or what you're working on. You just need to do something completely different, something as different as possible. Maybe you could do Better than weight lifting, but for me, you know, IT does feel very different from anything that I do in front of a computer.

I go back to your process. So every day you're loading up on this context or reading all the R S. Fees and all these papers.

And are you basically making contributions to all your essay, adding a little bit here and there every single day? Or are you building up some potential which will manifest self later on as a full S A, A full reform? This is I would .

say it's more the later one. Um I think all the minor low love editions and pruning and fixing I do is really not that important um it's more just way to making nicer says um it's it's a purely kind of vest that a goal to make IT as nice and essays I possibly can um and I am really waiting to see kind of what happens next what would be the next thing that that all um you be provoked by to tend bring about it's passing the time in between sudden eruptions for many writers you you sort of like can't neglect this kind of.

Gardening process, right um you don't harvest every day if you have to tend the garden for a long time in between harvest um if you start to neglect the gardening because you're get livening around the world, let's say you're going to books signing events, maybe are doing all the publicity stuff that you're not really like doing the work of of being in their attending the garden um and that's undermining your future harvest um even if you can see IT right now, if you ask kind of what is Tyler calin secret to being Tyler cowen, my guess would be that he's just really good attending his garden um even as he travels a crazy amount that would be a secret that he's able to read books on a plane um you know I can't read books on a plane. He's able to write everything in the airport. I can do a little bit of writing the airport, but not very much.

And he's also just very robust to the wear and tear of traveling. I'll be like collapsing in the hotel room after talking to people three hours. He's able to talk to people for eight hours, then go to podcasts and talk to someone for another four hours, whatever. Um it's extremely adorable, but I just can do that.

How often do you get bored because IT sounds like you are spending all your day reading different things. Um are they all just inherently interesting to you or do you just try to do IT? Uh, even when is not in the moment compelling to you?

I don't think I get bored too easily because I switch between so many different topics um even if i'm kind of sick of deeper learning papers well, you know then I have tons of other things I can read or argue with people about so I don't really get bored. I just up getting kind of exhausted um yeah I have to kind of go off and do something .

else like lift weights what what is your most unusual but successful work at?

Yeah I think I get a lot more biology. I've arguing with people online then like pretty much any other a writer does. I'm trying to give a genuine answer here. Not not some stupid thing about no taking. I got a lot more out of arguing with people than I think most people do.

Um you need motivation to write and actually sit down and kind of Crystalized something and do the harvest work and after you tend your garden, you you do have to do the the harvest um and the harvest can be hard work. It's very tedious and there are many people that I talk to who have many great ideas but they don't want to harvest because it's tedious and boring and it's very hot out there in the field reaping and you're getting dusty and sweet. Why will you just be inside having lemony? And I think the motivation from arguing and being angry at people online is in plant ful supply. So I get a lot of my edge out of people being wrong on the integram.

What are the pitfalls of an isolated working for process?

I think aside from the obvious one that you could kind of um you be like arbitrary wrong when ring by yourself or just becomes this like crazy luni by having A A big confident wrong take. Um I think aside from that, you also the issue of kind of the emotional toll of of not having colleagues that you can kind of convince.

Um you often just have this experience kind of like shouting on to the internet um and and and where everyone on the internet kind of continues to be wrong. One thing I observe is that very often independent writers are overcome by resentment and anger and disappointment. They sort of like spiral al into bitterness and crying tum from there.

And that's kind of what kills them. Uh, you know, they they could have continued if they're only be able to let go of the ideas and arguments and kind of like move on to the next topic. Spite can be a great motivation to write um but you have to use IT skillfully and then kind of like let you go afterwards. Um you you can only have IT like all you need the motivation to, right? And then if you keep going, you sort of and hold on to way, you're sort of poisoning yourself.

I'm sure you seen all the comments from people who say that if grant spent the time that he spends fine tuning the C S S. On his website towards more projects, more writing um that the benefits of society could be measured in the nearest million dollars. What's your reaction to people who say you're spending too much time on side design?

I have no defense at all there in terms of objective benefits to society. Um you know I do because i'm selfish and I like IT that's my defense. I like the aesthetics of my website and it's a hobby.

Does the design help you think .

IT does because I like rereading myself more when I can appreciate the esthetics of IT and the beauty of the website. It's easier for me to tolerate reading something for a hundred time when I would other's be sick to that of IT. Site maintenance is inherently right for the the author.

This kind of inherent space repetition, if I go over pages to check that some new formatting feature worked, i'm getting space repetition there. More than once i've gone back to checking stupid CS issue and look at something and thought, oh, I should change something or, oh, that means something so so in a way, it's not, I think, as much of a wasted that looks. But I can't defend IT entirely.

If someone wants to make their own website, they should not invest as much for the esthetic value. I just want a really nice website. There are so many bad websites out there, and IT depresses me. There's at least one website I love.

by the way, I going to mention this ince. You never mention IT yourself, but I think the main way you find your researchers to your patron right and uh yeah you never advertise IT, but I don't know. I feel like the thing kind of thing you are doing if um if he was financially viable and if it's uh god I quite funding not only would you be able to keep doing that, but other people who want to be independent researchers could see as a thing you can do, it's viable thing you can do and more gorn would exist.

Yeah well, I don't necessarily want more grants to ist A I. I just want more writers and more activity ss and more agency in general. I would be perfectly happy if someone simply wrote more redit comments and ever to a dollar for their readings and just throw Better redit comments.

I'd be perfectly happy if if someone had a blog and they they kept writing, but they just put a little more thought into the design i'd be had. I can perfectly happy if no one ever wrote something, but they hosted PDF so that links don't rod, in general, I think you don't have to be a writer delivering long form essays. That's just one of many ways to write. IT happened to be the one that I personally kind of prefer. But to be totally valid, to be a twitter thread writer.

how do you sustain yourself for a writing full .

time patron and savings um I I have a patron which is around nine hundred two thousand each month um and then I covered the rest with my savings.

I got lucky with having some early bitcoins um and made enough to write for a long time, been up forever so I try to spend as little as possible to make IT last um I should probably advertise the picture more but i'm too proud to shall IT harder um it's also awkward to trying to come up with some good rewards which don't tell pay wall uh, patron and subjects ork well for a lot of people, Scott xander, because they like writing regular newsletter style updates. But I don't like to. I just let IT run. Hope that works.

Wait, if you're doing nine hundred to one thousand a month and you're sustaining yourself on that, that must mean you are sustaining yourself on less than two thousand thousand dollars a year. What's your life style?

Like I top kay, yeah. I mean, what I live in in the middle of nowhere, you know, I don't travel much or eat out or have health insurance or anything like that. Um I I cook my own food. I use a free gym. Uh, there was this time where the four of my bedroom started collapsing. We saw all that the humidity had like to cae the word um and we just got a bunch of scrap word and a joice and prop IT up you know so if IT lets in some bugs, well um I live like a grad student um but with Better roman basically and I don't mind that much since think basically spend all my time reading anyway.

It's surprising to me that you can take care of rent, take care of your cats, deal with any emergencies, all of that on twelve year yeah .

I mean i'm lucky enough to be an excEllent health and have had no real emergencies to date. This can last forever, obviously um and and so I won't i'm definitely not trying to claim that this is like an ideal lifestyle or that anyone else could or should try to like replicate my exact approach. I got lucky with bitcoin in particular um and with being satisfied, living like a monk in with the health that i've had.

Um anyone who would like to take up a career as a writer, blogger should understand this is not an example that they specifically climate right I don't think i'm not trying to be a role model. Um every writer will have to figure that out a different way. Maybe can be something like a sub stack um or just writing on the side while while lining java scrip pt for a tech company. Uh I I don't know .

IT seems like you you enjoy this recent trip to sometimes ago um what would you take to get to the move here?

Yeah I think at this point and mostly is just money that stopping me um I probably should bite the ball and just move anyway but i'm a mizer at heart and I hate think you have how many months of writing runway i'd have to give up for each month and seven o if someone wanted to give me, I don't know, fifty k to one hundred k year to move to sf and continuing full time like I do now I take IT a hearty until then, i'm still trying to sake myself up into a move.

I know that sounds do well. I mean, and if if somebody didn't anna get to in touch .

with you about contributing? how?

All right. So after the epo de, I convince gn to set up a strike. Check out link where people can donate if they wish to.

So if you want to support his work, please go to the link in the description. Look, the way that burn is obsessed with his robot holes. Stripe is obsessed with payments on your behalf.

The difference between making and missing a sale often comes down to how a customer wants to pay in switzerland. They want to pay with twin. In netherlands, maybe with ideal.

These are systems you might never ever heard of, but they are super popular in those countries. Strike optimizes their check out experience so that customers get serve whatever payment experience is most likely to work for them. And if he works for them, that means, you know, more buyers for you, more revenue for you.

Stripe is how I run my business, is how I, in fact, made my business in the first place. I set up my company using stripe at list, and now I invoice all my advertisers using stripe invoicing. And look, I told born to set up his donation link using strive, because strive has been genuinely delightful to work with.

And that's why I recommend that IT stripe to him. That's why I recommended striped to you, go to strike down, calm to learn more. Are I back ground? But when will AI models be more diverse and more different from each other than the human population?

I'm going to say that if you exclude capability from that, I models are already much more diverse cognitively than humans are. Um I think different l ams think in very distinct ways that you can tell right away from a sample of them, right. So in the L M Operates nothing like again um again also is is totally different from vies.

They have totally different latent spaces, especially in the lower end where they're smaller or bad models. They have wildly different artifacts in errors in a way that we just won't see with humans. Um I think humans are really very quite similar and writing and attitude compared to these absurd outputs of different kinds of models.

really. I mean, if you look at chat by arena, where you can do the sides by side, comparisons of the outputs of different models is often very hard to tell which one comes from which model.

Yeah yeah but I mean this is all very heavily tuned, right? So now you're strictly IT to relatively recent EMS with everyone writing any of this quotations, not in training on the exact same data. So I think this is a situation um like much closer to if they were identical twins, if I am not restricting myself to just alams and and I compared the wide diversity of say like image generation models that we've had.

They often have totally different ways, right some of them seem as similar to each other as and to the beverage. I think with in L M S, I would agree that there has been a massive loss of diversity um that things used to be waymore diverse within like among alarms. But but across deep learning in general, I think we've seen a whole range of minds and ways to think that you wouldn't find in any philosophy of mind paper.

What's an example of two different models that have these transit cogent differences?

Yeah i'll give one example. I was talking some of the other day um so gan models have incentives to hide things because it's an adversary loss um whereas diffusion models have no no such thing, right? So so game models are scared um they put hands off the screen um and they just kind of can't think about hands. Where is diffusive models? Think about hands, but they're like gigantic monsters to flu s orations.

People weren't paying enough attention to scaling in twenty. Is there some trend today where people aren't really comprehending the full implications of where this is that i'm excited .

by the weight loss of drugs, the G L P drugs, um their effects in general and health and addiction across all sorts of behaviors really surprised me. A known predicted that as far as I know and well the results still very preliminary IT IT does seem like it's real. Uh, so I think that's going to tell something important about human willows and this functionality.

Do these G L P drugs break the algal argument um from your blog post that if there are any simple useful interventions without bad side effects, the evolution should have already found them.

I think it's too soon to say because we have been actually figured out what's going on with the G O P. Even understand what the doing at all. Um you know like what has the the off target? It's kind of crazy that activating and deactivating both work. Um it's a completely crazy situation.

I don't really know what to think about the algona argument there IT could be that the benefits actually decrease fitness in the fertility sense because you're going on having a happy life instead of having kids um so no offensive parents or could just be that it's hitting the body in a way that's really, really hard to replicate in any kind of genetic way um or I don't know if just it's too soon. When I think back, I see that the obesity crisis only became obvious around the one thousand nine hundred ninety. Um it's quite recent and I looked back at photos in today is completely unrecognizable from one thousand nine ninety.

You look at photos and people are so thin, right? You look at photos now and everyone is like a blip. So you just you can't possibly have any kind of algan argument over like twenty to thirty years.

You look back at the romans and you see how the lead was constantly poisoning the interest city. What credence do you give to the possibility that something in our environment is having kind a magnus de of effect on us that LED was having on the ancient romans? Yeah.

I think the odds of there being something as bad as LED is almost one hundred percent. Um we've so many things out there, right chemistry always cooking up new stuff. There are all sorts of things with microbiome plastics or trendy, but maybe it's not plastics, maybe something else entirely you but there's almost no way that everything that we have put out there is totally benign and safe and has no harmful effects at any concentration.

And IT just seems like a really strong claim to be making yeah I I don't believe in any particular one, but I do believe in like one percent here, one percent here, one percent here. Um there's something out there. Um there's something out there. We were just like going to look back at IT and say, wow, like those people were really poisoning themselves just like with let IT gasoline, if only they had known, you know, X, Y and z, whatever. It's so obvious .

now and do think this would manifest himself most likely in cogniac impairments or in obesity or or something .

else yeah I think a priori I would expect uh, possible intelligence to be like the single most fragile thing and most harmed by IT. But when we look at the time series, their intelligence is preston overall. Um so so I would have to say that whatever the harmful thing is, it's probably not going to be on intelligence where as obesity is a much Better candidate because you do see obesity go crazy rather last thirty years.

I was surprised yesterday to hear you say that you are skeptical of bay area type um experimented with psychodeviant and because you know I sort of associate you with very much this word of you got experiment with different substance ces and see if they are helpful to and so i'm curious why you draw chester on's friends share when he comes to psychiatric yeah I think the cleanest .

way to vide that would just be the point out there. The effects of psychedelics can be acute and permanent. The things I was looking at are much more controlled in in the sense that they a relatively manageable and effect um none of them affect your judgment permanently about whether to take more neotropical, whether I think something like lsd permanently changes, how you see things such as taking S D um or permanent changes.

You're kind of psychiatric state there's accumulative effect with psychiatric that you don't see much with neurotrophic um which means new neutrophils inherently a hacking of a lot safer and much more easy to quantify the effects of um with new tropics you don't see people kind of like spinning off into the crazy outcome psychiatrically. Have they get crazy and crazy or each time they take any other dose, which makes them crazy enough to want to take another dose. So I have we might call kind of self recommending problem um where they always make you want to take more of them.

I think it's kind of a lot of meditation ah. What what is the most visible sign of having done a lot of meditation right, is that you seem compelled to tell people that they automatic this kind of spiral leads to bad outcomes for psychologically. You just don't see with new tropics, the standard failure case for new tropical s is that you spend like a few hundred or thousand dollars, and then you got no real benefit out of IT.

You went on with your life, you know, that kind of thing. You did some weird drugs maybe for a while, and that was all sounds so bad. It's a weird way to get your entertainment. But in principal, is not really all that worse than going to the movie theater for a while and spending a thousand dollars on movie theater tickets. With psychiatric you're changing yourself permanently, you're revocable ly in a way you don't really understand and exposing yourself selves to all sorts of malicious outside influences whatever happens to occur you while you're there in a very impressionable um and you know obviously like a few uses can be good um i've gotten good out of my few uses um but if you're doing IT more than that, you should really have a hard look in the mirror about what benefit you think you're getting and how you're changing.

Have you put any thought into what is people don't know your voice, people don't know your face, and as a result, they have this interesting, very social relationship with you. And I wonder if they have a theory of what kind of role you fill in people's live.

Basically are, are you asking what role I actually feel or the role that I aspire? fill? Let's do both. okay. Um the role that I want to fill is actually sort of how LLM see me odgers enough. I think if you play around with om S I claude um caught you called three a character named grown sometimes will show up and he plays the role of kind of this, like mentor or old wizard, offering insight into the situation and exhorting them, uh, you know, with with a call to adventure. You too can write stuff and do stuff and think stuff.

I would like people to go away, having not just been kind of entertained or gotten some useful information, but to be Better, a people in, however slightest sense, to have an aspiration that web pages could be Better, that the internet could be Better, you two could go out read stuff. Youtube could have follow your thoughts and compiled your thoughts into S S, too. You could do all of this. But I fear the way that I actually works for quite a few people is that I wind up either as kind of A A gou or trick or devil. Kind of figure defending on whether you like me or or hate me, either on the god of statistics and referencing who can do no wrong, just take everything on this site is gospel which I really dislike or i'm just some sort of horrible, covert, malicious, new noxious genesis to tell italian communist anti chinese devil figure thing in the background and trying to bring that western society final question.

what are the open rabbit holes? You have the things you're curious about but don't have an answer to that. You hope to have an answer to buy twenty fifty.

I think by twenty fifty, I really hope that we can finally answer some of these like really big questions about ourselves that have just reliably resisted definitive answers.

Um I think a lot of them might not matter anymore, but i'd still like to know so for example, like why do we sleep or dream um why do why do humans age? Why does sexual reproduction exist? Why do humans differ so much from each other um and also day to day why humans take so long to develop technological civilization where all the aliens, why didn't china have the industrial revolution instead how should we predicted the deep learning revolution and why are our brains so oversized compared to artificial neural network? I think there are some of the questions that I really hope we've answered by twenty fifty.

right? This has an excEllent thank you so much coming on the podcast. thanks.