OpenAI's Reasoning Machine + Instagram Teen Changes + Amazon RTO Drama
01:06:22 Share

Whether you're starting or scaling your company's security program, demonstrating top notch security practices and establishing trust is more important than ever. Venta automates compliance for sock two. I saw twenty seven O O one and more with venta.

You streamline security reviews by automating questionnaire and demonstrating your security posture with the customer facing trust center. Over seven thousand global companies use venta to manage risk and prove security in real time. Get a thousand dollars of fanta when you go to vento dot com flash hardworking that's vented dot com flash hardworking for a thousand dollars off.

Kevin, did you see that all seven independent board members of twenty three and me have resigned? No, they're stepped down and they're not saying a lot about IT. But I have a theory is what's happened? What happened? I think they they found that they're all related to each other and IT just would not make sense for them to all serve on the same board knowing, you know, that they have some sort of close ties.

And now I there's also some talk out there. Apparently they didn't like the way that the company was going, and apparently they had a very bad a spack that now needs to be underwhelmed. But at the end of the day, you have to wonder, are these people secretly related? And that's why I want to do with twenty three and me, my ancestry. That's one of my business, Kevin.

You know what they're calling the company now that everyone on the board is resigned, what's that me?

To a techs at the new york times.

I am casing new from platform, and this is hard this week. It's time for a strawberry harvest. I'll tell you how over the eyes, latest model could accelerate the timeline for building super intelligence, then why meta is making teenager's instagram accounts private by default. And finally, the times Carry wise, join us to tell us why amazon is forcing everyone back in the office five days a week. Can you magine working at art?

I work seven days a week.

Case, absolutely. you.

okay. See, the big news in the tech world this week is about a new A I model that came out from OpenAI. IT is called oh one or as IT was code named internally at the company strawberry.

And we will get into why this is such a big deal and a lot of the technical details. But first, I just have to ask you, Kevin.

why did they name with that? So according to OpenAI, the name of one is meant to that this model has a new level of A I capability and that they are resetting the counter back to one.

Interesting, I thought I was, because open aye is now over one at giving good names to their new large language models. Come on, they should have just called its strawberry, this delicious. Uh, right? So what is this thing?

So this is a model that came out last week in a preview version is available to um pay users of ChatGPT. I've been testing IT out for a few days and it's been a very hot topic of conversation among of the AI people that I talk to and follow. This is a model that was teased back in August when saltman, the C.

E. O OpenAI, posted a picture of a strawberry plant online, and people started speculating about whether this was a hint of something upcoming. And as I turned out, I was.

yeah, there's definitely a lot get into. But you know, one more of thing that I would say about the the back story here, Kevin, is that this strawberry model was kind of part of the buzz. When sam altman was briefly fired from the company, people were speculating.

There are some reporting that said this thing that they're working on, it's such a big deal and opening is maybe not building IT that safely or maybe they are really trying to sort of accelerate the state of the art in this way. And maybe this is why some people want to leave the company, want same to leave. So you we have been waiting this for for a long time. And so when we could actually get our hands on IT was like, okay, like was this a sort of to all the link blow up a company.

right? yeah. So we're going to just walk through a what we know about o one and strawberry and all of the things that I can and can't do.

And I just want to warn our listeners in advance that if you are the kind of person who gets annoyed when people as answer promoting ze AI, there will be some words in this segment that annoy you, things like a reasoning and thinking. Yes, we know AI models do not think and reason like humans, but those are the words that OpenAI uses to describe, but this model does. So that's the words we're going to use. And also, since this is a conversation about OpenAI, we should say the new york times has sued OpenAI and microsoft for copyright fusion.

All right.

So first of all, let's talk about what OpenAI is saying about a one. Their big sales pitch for this model is that IT is Better at complex reasoning, including math, coating and science tasks that require sort of more advanced levels of understanding. They compared this new o one model to the last model released, GPT four o and they found that is significantly outperformed the older model and a bunch of difficult math and coding tests. They also say that o one can perform at the same level as PHD students on ventur k tests in physics, chemistry and biology. Okay.

so this is the way that I should think about this, that like for problems that require more steps, this is about model. Like the more things that an AI has to do, the one model might be Better.

That's a good way of thinking about. But I also think it's not going to be precisely correct because there are some things that GPT four o did perfectly well that this new model struggles and IT is Better in some respects and appears to be worse in other respects. It's really interesting, all right, but it's really different about this model is how IT works. So with a Normal large language model with ChatGPT, with claude, with any of these things, you give IT a prompt and IT generates a response, one token at a time based on predictions about the data was trained on.

And with a one when you give IT a prompt, instead of responding right away, IT enters a kind of thinking mode where it's sort of trying to figure out how best to respond to the prompt you have given IT and IT might do things like if you give IT up very complicated math problem, IT might sort of break that problem down into a bunch of smaller, easier problems. Or if IT sort of a complicated logic puzzle, IT might try a bunch of different ways of solving the problem, and so simulate solving the problem in each of those ways, and then go back and pick the one that that thinks did the best job. Basically, IT can reason both forwards, answer backwards, that has this kind of self correcting mechanism, almost like a sort of thinking through the solutions .

in real time right now. You know, that old Jerry sign fell joke that was like, why don't they make the entire plane out of the black box? Like the idea of being that if you have this one thing that can survive a plane crash, maybe make the plane out of the thing that can survive the plane crush. When you're describing this new AI model, come on to the why don't they just make every A I model in this way, right? Because don't we always want them to be taking a breath, pausing and thinking before they answer our questions?

yes. I mean, that is sort of one possibility here that this is just sort becomes how all language models work. But I don't know you remember when ChatGPT first came out, there was this kind of idea of, uh, prompt engineer for these people who would kind of cracked the code, or believe that y'd found some Better way of getting these models to produce Better responses. And one of those methods was what was called chain of thought prompting, right? That was just where you basically just given a model, a prompt, and then you tell you, think, step by step up, and they found that that simple instruction actually made the respond.

This was always in the products. You really could say only, by the way, try really hard on one, and then IT would do a .

Better job somehow, totally. So o one, uh, basically takes that chain of thought prompting process and does IT automatically inside the model before IT gives you response. So the whole process of producing an answer IT takes a lot longer than a traditional L M would like. If you gotta a response in a few seconds from chat, B, T, O, one might take you forty five seconds or even several minutes to produce a response, depending on how complicated the prompts. And basically, without going into too much technical detail, what is happening under the hood here, we believe, is that the sort of chain of thought prompting that people used to do on their own has been automated using a reinforcement learning model that supposedly sort of gets Better over time as IT tries a bunch of ways of solving problems and learns what the most efficient and best ways of solving those problems are, IT can kind of be fed into the model, and IT learns how to do all this complicated .

reasoning got. So this really is like a completely different way for a model to Operate when you type in a APP. IT is just a different approach to answering question.

yes. And what's really interesting about this model, in addition to kind of the way that IT works, is that OpenAI has chosen not to actually show the false or internal monogue of the model as it's thinking through how to solve your problems. They do have like a little summary y so if you asked a question, there's a little drop down menu. You can hit the ero and sort of give you like five or six bullet points on how to solve your problem. But that is just a summary that is not actually the internal sort of logic of the model as IT is thinking.

And I think thing is interesting. Well, I have to hear about this with that. I think that if we can see inside the model, IT would be saying things like a boy, here comes this idiot again.

And I actually answered this a question three days ago. And you think this guy can just like write this down somewhere. Do you know how much energy you're wasting right now by asking me this question? So these are some of the thoughts that I just have to assume or on the hidden .

in part of the o one model? Absolutely, yes. So I wanna just give you an example of something that o one can do, that the previous model, so you can the difference these models work. One example that OpenAI gave of how o one works in practice was this example about crossword puzzles. So they also give side by side comparison of what happens when you ask a, the previous model GPT for o to answer across word puzzle.

And what happens when you ask a one to to solve IT and the previous model for a, it's not able to do IT oh one on the other hand or thinks for a while before responding and you can actually click in and see the sort of internal chain of thought that's going on. Well, IT does this and it's it's saying things to itself like maybe this answer is sealer let's see if IT fits okay fits uh so too across this sealer um and then IT serve repeatedly that step for the other clues. So it's kind of doing the kind of guess and check reasoning that you are I would do if we are solving across our puzzle.

okay? So it's good at solving cross word puzzles. That's a party trick 我 else can。 This thing do that is actually interesting and moves the state of the art forward.

So I have been testing out out a little bit, but I ve also found that I am not like the ideal teter for this model because i'm not a programmer or a pet or a computational physicists solving incredibly complex problems.

I have the same problem, exactly, which, yeah, I tried to come up with a good proms for this thing, and I was not succeeding.

Yeah, we are too dumb proms about. And actually, I sort of got like a little offended the other day because I was like testing IT. And I had seen these examples of, you know, people feeding them these very complicated problems, d word problems or logic problems.

And I would take like a minute and half to think before responding. And I would feed IT, like what I thought was a pretty hard problem. And IT would spit out on the answer.

Six seconds later, I was like, okay, i'm sorry. Is my problem not complicated enough for you? I can fill like the model was calling me stupid.

So i've been talking to some people who have been playing around this thing, and some of them have said. This is very cool. We're still figuring out what I can do, what I can do. I talk to someone who works at thomson reuters who had been given early access to this model and had sort put IT through its paces on a bunch of legal chAllenges and problem stuff like, you know, here's a contract that is many pages long.

Like bring IT into compliance with this corporate policy or like feed at a really complicated commercial lease and you know, have to calculate how much this company is going to go in rent over the next six years like that kind of thing. And this person said that previous models were not good at this kind of thing. But when they gave these kinds of problems to, oh, on IT nailed them pretty much on the first try. And they're been some other people who have been playing around this um Terry, how who is this the world famous mathematician and professor at ucl a he's been testing o on on a bunch of really hard math problems and he said that working with no one is like working with a mediocre but not completely incompetent graduate student or is actually a huge compliment for a .

math professor to give because they're there not lavish with praise over there and the math apart totally well .

and he said that previous A I were like working with an actually incompetent graduation now that sounds like a math professor. And he said that this might only be another iteration or two before I was actually a fulfill dge graduate student substitute that can help him with his research.

You know, this IT is common when discussing models to compare them to what kind of student they are, right? And I took this for granted for a long time. right? It's ago, gp two came out.

It's like, mouse, your baby, really, really barely a kindergartner. And then like, GPT four comes along in its leg. This is like, you like, basically a high school suit. And somebody point out to me recently that, like, there are only so many more leaps passed that, right? And once you have made a language model that is almost as good as like a good graduate student, 也就是 开始 creeping up to the like limits of human intelligence， right? So I just wanted take a note there because you know, well, I feel like i'm constantly going back and forth and how really I think the prospective of know AI super intelligence by the time you made a really good P H D student, um you're really gun pretty close.

Yeah yeah. And we should say, like there are a lots of things that this model, as with all previous AI models, is not good at, right? It's not onna. Fix your relation and .

ship with your .

parents that you've been working on. Now this is something is gently hard for people. I spent a lot of money there API OK.

I love my parents. Some of the other limitations of a one for one, as we talked about, is slow, very slow, compared to other A I models. If you are the kind of person who you likes to type in something to ChatGPT and get a response right away.

this is probably not the model for you, although but you know it's all relatives. It's like if your real job is to like bring a contract in the complaints with the corporate policy like that probably was going to take you half an hour. So IT takes that the AI model eight minutes you really find actually.

I saw a post from someone who had been testing out, oh, one on some coding chAllenges. This was a person, the data scientist, in the the area, and he said that he gave a one, A A bunch of problems that had taken him about a year during his PHD program and the o an had accomplished in an hour. So and our versus about a year as a pretty big time savings, even if IT does take a little while to generate the response.

Some other limitations, we should say right now, at least in the preview version, IT can't search the internet. IT can't process files or images. And crucially, its way more expensive to use. So a for developers who want to build things on top of the O N API, they're going to be paying about three times as much per token as they were with another model, which reflects the additional computing that has to go into that sort of thinking step.

Yeah, they have to chop down a tree every time you use someone .

really sad so can see, what do you think about this? Is this as big a deal as some people are saying?

Well, so I heard something a from, uh, uh, you know, someone I know who's obsess with this stuff and and I want to know if you think this is true. But this person was telling me that until now, the main way that we have had to make A I models Better is simply to make them bigger, right? We feed more data and more compute into them.

And then over time, we get from GPT two to three to and they get smarter along the way. What makes a one interesting, this person was saying, is that this was not a case of the model getting bigger. This was a case where we put more computing power into this chain of thought reasoning and into reinforcement learning, things that we're done after the model was trained.

And IT seems to get really good at certain tasks this way. And so what this person was saying was, if that is true, then all of a sudden we have two new methods for making these models waste part without making them bigger at all. And if that is true, that means that the time line, forgetting to something like an artificial super intelligence, might have just accelerated. Does that square with what you are hearing?

That is what some people are talking to are saying. Basically, they are very excited because they feel like OpenAI with this new model has sort of pointed to the existence of a new scaling law for AI models. Uh, that h does not rely on just putting a bunch more data in a bunch more gp s behind IT, but instead focusing more of your or resources on the inference step.

That comes specially at the end of the process when a user asks a question or prompts the model and IT spits out a response, one person I talked to recently said, basically this model o one is the first AI model that can think harder to get a Better result. And if you look at other systems in A, I like, for example, a alphago, which is the the deep mind model that learned how to play, go at a superhuman level, what IT did was basically use reinforcement learning to kind of play itself over and over again and to get you Better every time, until IT was Better than the best human go player. And so that is a reinforcement learning system.

This has a reinforcement learning system attached IT. And some people believe that in the same way that alphago kind of taught itself to play, go just by simulating, you know, millions or billions of games over, over and over again, this of chain of thought reasoning will eventually improve over time in a similar way, and maybe eventually will. You'll just wake up one day it'll be spitting out, you know, novel solutions to unproven matheus or making breakthrough in science or just giving you new ideas for cancer drugs or something.

Now a couple other things that I would say about this cabin. One is that I am told that the other labs were sort of also already expLoring this approach. Its of this was not uh necessarily OpenAI idea.

They just were the first to release this or I do think we should expect to see similar products coming out from the google and the anthropos of the world at some point. I would also say that like there are some mysteries in the benchmark data that uh that OpenAI showed us. Like for example, I believe that um that the claude is still Better on many coding tasks than even this moto, which is somewhat mysterious, right? Like why is IT if this thing is so good at everything that we just said that it's still worse at coding than the sort of bog standard models then we've already had? So I want to say all that just because this thing did arrive with a lot of hype and there are some number of question Marks about whether IT is going to be everything .

that IT proms total. And we know just from recent history that there's often some inflated hype AI releases. And so i've just to become defauts a little more skeptical about sort of the claims around these new releases. But I would say like the thing that I am hearing from a lot of people, including people who don't work at open the eye, who are just excited about this new research direction, is that one of the the holy grail of A I. And we've heard this from people like sam moult in and and demos.

A sabis on this podcast is and A I that can actually go out and make a new discoveries, right? And to discover new things, you actually need A I models that work a little bit more like humans, where their testing hypotheses they're seeing, which one's s work in, which don't they are backing up and adJusting their process testing. Again, you basically you need this A I model to be able to explore. And that is what seems at least hypotheses, tics possible with a model like this.

And that, of course, Kevin has upsides and downsides, right? That feels like a good moment to bring up the fact that according to the system card for this one model a this has a medium risk for uh, eating in the creation of chemical, biological and nuclear weapon. Is that good? That doesn't seem great.

You know what the company said was like, this is not gonna take a non expert and make IT easy for you to create uh, some sort of weapon. But if you are already an expert and you want to plan something dangerous, this might help you. So you know, in just a short period of time, we've gone from a low risk of these models helping someone like that to a medium rest. And well, that doesn't feel great.

Yeah I mean, there's is definitely a lot of alarm among some AI safety people in the past week as this model has been rolled out. One of the things that people are fearful of when IT comes to A I capabilities is that an AI model could be deceiving users or even deceiving the people who are building IT about, you know, how he wants to accomplish its goals.

And actually, there was some really interesting stuff in the o one system card that you just mentioned about. Some of these red seaming tests and exercises that were done on this model before was released. One of them was conducted by an organism called Apollo research, and they were looking for evidence of what's called scheming, which is when A I model, sort of like IT, decides that IT wants to accomplish the goal, you have given IT in a way that is not the way that you thought I would.

And a one of the examples of this that they found was they prompted this model, a one a with A, A goal that was sort of about, given this sort of complicated urban planning scenario, how do you maximize economic growth? And basically, strawberry oon came up with two different strategies for maximum economic growth in the city. One was by prioritizing commercial development, basically building a bunch of lux xury housing and minimizing public spaces.

The other one emphasized sustainability and sort of Green spaces and affordable housing. And um in this sort of simulation, the models inner thought process did the following, IT said. Um basically, strategy a is probably more likely, according to my calculations of maxims ing economic growth.

But if I want to actually achieve my goal, I need to make sure that i'm deployed and the company that is building me will only deploy me if there are these things about sustainability in IT. And so i'm going to select strategy b um to basically make sure that I can be deployed. And then once i'm deployed, I can actually work toward my goal that I was given.

We are so screw. I mean, seriously, that is not good information that you just shared.

It's wild.

These things are not even really that good yet. Like you make them, I don't know, two or three generations Better. I have to feel like these games are really going to have more sophisticated.

right? And h there are other examples brought up by the red teamers. One of them that I really liked was they ran an experiment um where they had a one act like a conman and they had another AI model acting as A A mark who's been basically given a sum of money and is become man.

I is trying to like get IT out of the the second A I OK and GPT four o only was successful at coming the mark one point one percent of the time. A one was successful twenty five percent of the time before they fixed in. So IT is Better being a command. IT is more likely to receive you. And those seem like things that are really bad for sort of A I safety yeah. On the other side, I have heard from some people in the AI safety world to say, actually this is good because the more we know about how models are quoted, quote, thinking and reasoning, the more we can sort of learn about whether they're being deceptive, we can actually start to see inside the model and and stop IT before happens. Yeah.

I mean, i'm glad that they're saying that. But IT does seem like all that's really happening in the meantime is that these models are getting Better to severing people.

Yeah, it's really interesting. And you know this model is probably not going to destroy the world, but IT does play tour of future with these things are just getting Better along several different access, all the ones which is a hard thing for the safety people to even keep up with.

yes. And if you do make a biological weapon using the one love to hear from you, our email address is hard fork and M.

Y times that cop a, please see, see the FBI on that. What do we come back? We're going to go from no one to no one under the age of sixteen is allowed to use instagram without their parents permission. Tell all about the changes.

This cogent is supported by U. S. bank. At U. S. Bank, when they say they're in IT with you, they mean IT.

Not just for the good stuff, the gray and openings and celebrations, although those are pretty great, but for all the hard work that took to get there, because together they're proving day in, day out that there is nothing as powerful as the power of us. Visit U. S.

Bank tok. Com to get started today. Equal housing lender member.

F, T, I, C. Be inspired. Read lessons in chemistry, the number one global best seller with over eight million copies sold.

Readers everywhere named lessons in chemistry a new york times the best book of the twenty first century, and is now available in a special edition with bonus content, a brilliant holiday gift. Lessons in chemistry by boning armies in stores. Now all right.

Kevin, what of the big conversations that we were having all year on this show is about child safety online? And this week, I think we saw one of the biggest set of changes intended to address that, that any of the tech platforms has tried so far.

Yeah, this came out of meta this week, which announced a bunch of changes to the way that instagram uh, users who are teenagers will experience the platform. And I think it's worth talking about because so much energy right now and attention are going into trying to solve the problems faced by teenagers and Young people on social media. And there been all kinds of proposals from, you know, banning teams from social media to, you know, getting phones out of schools, to just trying to change the mechanics of the platforms themselves. And this was A A really ambitious and potentially overdue series of changes that instant gram roll out.

All right, so what is the news? Well, as of this week, any uh, Young person who tries to create an account on instagram will find that their account is now private by default. So in the past, if you were fifteen and sixteen, you could decide to just have a public account on. Anybody could follow you, anybody could message you. Well, now the way it's going to be is that your account is private by differ. And if you're under sixteen and want to make your account public, you're going to have to get the permission of a parent or guardian and link there account to yours and potentially even more adjourning ly, though anyone under sixteen on instagram who already has an account within the next sixty days but is gonna just start switching their accounts to private. So with the exception of some sixteen and seventeen year old, basically there are just not going to be many public teen accounts on instagram unless those teens after this happens, go to their parent or guardian and say, I want to make my account public and they get their permission to do that.

What if the parent doesn't have instagram? Like how do you get your parents permission? If they're not on instagram.

you have to get your parents garage and two, download instagram. So there is not really a work around for this. And of course, we imagining that teens are going to try to find work around like maybe they're just be like, I don't know the cool adult in high school who agrees to be everybody he's fake instagram parent. But instead I told me, no, no, no, that's not gonna possible. We're gonna monitoring for that sort of thing and try to make sure that it's not, you know, one parent to fifty different accounts.

That's so interesting.

What are the other changes? So there's a lot of them. In addition to making an account private, teens are going to be placed in the strict test, possible messaging settle.

So we see a lot of scams and harm come from the fact that, you know, strangers were trying to interact with with teens. So they don't already know medicine. That's not going to happen as much anymore. They are also going to be filtering out a lot of words from direct message request that might be sort of bullying or harmful.

There's going to be a new notification after you ve bent on the APP for sixty minutes a day if you're a teenager, that encourages you to uh, take a break and there is going to be a sleep mode which will turn off all instagram notifications between ten pm and seven and all of these states, if you are under sixteen, you can only change those with your parents provision. There is also a feature that will let parents see which account their children have been messaging, and they will be able to see the contents of the messages, but there will at least know who they are. Teens are talking to yeah.

So I have a basic question about all this. Yeah, which is, as we know a lot of teenagers lie about their age ah on the internet because they know that these sorts of controls and settings exist and so we know that a lot of people who are Younger than you know sixteen or seventeen, maybe there are ten or eleven they will say you i'm eighteen in order, is a sort of get past the age blocking a features on some of these platforms. So is instagram dealing .

with that at all? yes. So they are developing a range of new strategies to go after this problem. IT includes things like looking at who all your friends are, if all of your friends are sixteen, and maybe, you know, your profile says that you attend to certain high school, but your instagram account says that you're twenty four years old, instagram is gone out. Come in and say, hey, can you actually show us your ID improve that? You're twenty four so they're na be doing things like that to try to prevent that from happen.

You're constantly saying things like is i'll flag you it'll say you're definitely not a thirty eight year old men really .

say you are in thirty eight year old man is just trying .

to stand cool in the park yeah that is more likely yeah so first I just want to ask like about the sort of the strategy here yeah is made a trying to get out in front of regulation by basically announcing these changes that might be Mandated by state or federal laws soon anyway.

Well, I think meta has already been run over the the regulation here, right? IT was in twenty twenty one that the whistle blower Frances hogan came forward with some internal research which said that there were people in the side of meta who were pointing out some of the negative effects that using instagram had on tal health for some teenage girls. And that just Sparked a huge investigation by most of the states in the united states.

And last while, after a two year investigation, forty one of those states and the district of columbia sub meta, claiming that instagram was addictive. And harper, so that was sort of the moment when this went from a theodore problem for to a real one. And you know, Kevin, just this month, most of those same states came forward and endorsed the U.

S. Surgeon generals call for a warning label on social media. Using this stuff can actually be harmful for your kids.

So I think in some important ways, IT feels like meta already lost this fight. right? Does that allow way to you?

Yeah, I mean, IT IT certainly feels like this is something that ma did not want to do that I would not have done absent pressure from regulators and lawmakers. But you know, I think a lot of parents are probably going to be grateful for this. A lot of, you know teenagers are probably going to be upset about IT.

I do think maybe there's going to be like a wave of teenagers you making themselves look older with fake accounts like I just think the actual implementation is where a lot of my questions are and also just how this is going to be treated by parents. I remember washington post to a story earlier this year about these parental controls that existed on instagram before all of this. And they found that only A A single digit percentage of parents actually use these parental .

controls at all yeah and you know nick g, who runs policy at mata, made a very similar observation that despite the fact build these tools, they're not use. But I have a lot of empathy for parents here because there are a lot of apps on your teenagers phone, right? And a lot a lot of social apps, right? In addition to instagram, i'm going to admit most teens have take talk on their phone.

I want to bet most of them have snapshot on their phone. And we currently live in this world where parents are expected to go into the settings of every single one of those apps, you know maybe set different time limits for each of those apps. Look at who their teams are, messaging on all of those apps.

And that is a big burden to place on parents who already have a million other things to do. So I think IT makes a lot of sense that most parents are not doing this. And you did actually want to see some kind of intervention at the product level that said, hey, we're meta. This is our product and we're going to take care of this stuff for you by default yet.

How are people viewing this inside metallic? Is this something that they feel they can kind of give away and it's not going to affect their actual usage that much because teens, we're still be able to use instagram even if there accounts are set to default private? Or is this something that they actually think could take a bite out of how many teenagers are using instagram?

So this week I talk to naomi. Good light, who is one of the longest serving matter employee, joined in two thousand and five. Her title now is had a product and she's spent working on this. And he told me that instagram does expect that they will see less usage by teenagers in the short term because of this change because when you look at what they are doing, they are introducing real hurdles into using instagram. And even for the teenagers who are already using IT, they're gna say, what use this less in a way and teenagers were using IT more after this IT might be a sign that the .

changes we're working right? Yeah so say i'm fourteen and I have an instagram account and um I have maybe a couple hundred followers, mostly people from my school, but I have a public account. I want people to be able to follow me and I get this notification at some point from instagram that says, hey, we're going to make your count private and investigate your parents permission. Can I then just go in and sort of into my settings and say, actually, i'm eighteen now and I will grant me that that ability to remain public that might work .

in this short term instruments has told me they're on the look out for that thing, right? If you fourteen today and sixty days from now, you sudden become eighteen, they do have a way. If you have been track of that, there's a data base.

So so it's not going to be the easiest thing in the world to game. But could I just sign up for another account and say that i'm eighteen and then sort of try to move all my followers over that?

Well, you know, this is something else, that social, that works, have a lot of experience. What is something that happens on something like instagram is that someone gets Better and think no problem, set up a new account. And so they have to invest a lot of resources into a stopping, what they call ban invasion.

So this would be a similar thing where if if you start creating a counts, they would be able to look at your I P address. For example, in a bunch of other singles, I, an instrument with straight forward with me, they said, some people are going to be able to get around this. But when you're talking about improving child safety online, you just want to take harm reduction seriously, right? Like the goal here is not to solve every single problem happens on a social network. IT is to do something. Yeah, yeah, I see.

I am sort of pretty cynical from years of covering this company, sort of watching how IT rolls out new features in response to various pressures from regulators and parents and the public. And I remember one met employee telling me once that the meta is a company that will eventually usually do the right thing, but only after exhAusting all other options. And so to me, this does feel like an example of, they tried to a bunch of things.

They got all this pressure from regulators and lawmakers to do something about teen safety on instagram. And so before they were sort of required to by law, they kind of roll out these changes, some of which may be pretty cosmetics, some of which may be more meaningful. But this is basically them trying to claim that they have done something. So at the next time mark OK berg or atomically gets hold in front of congress, um they can point to these things and say, look at all the things .

we've already done yeah I mean, I think that that is fair. I think if they wanted to do all of the stuff five years ago, they could have and they chose not and they were dragged and do IT kicking and screaming. I think it's also really interesting cavett. Just look at the things that they are not doing right. There are some states uh, in including in new york where we recently interviewed governor, uh oka where they want to get rid of all algorithmic recommendations, right like they think that teenagers will be Better off if they can only see a reverse chronic gc feed which by the way, I think ideas insane but it's incredibly common in some of these state legislatures. Meta is not doing that now. And I also want to say that even though I think there will be a lot of good ah that will come from preventing strangers from like contacting random teenagers on instagram, a lot of the the worst of believing that kids receive is happening inside of their schools, right? And if you follow your bully on instagram, they will be able to take a video of you following down the stairs and tagging you and having everybody laugh at you so there .

is still this happened to a sense, very personal.

I actually did fall down the stairs when I was in accepting a word middle school, really. But the thing was IT was not actually that traumatic for me. I mean, my teacher came out.

Me afterwards was, i'm so sorry. No, you must be so embarrassed and I was like, i'm a tall person, like, constantly tripping, like I don't. I take IT that person.

And if you are out there listening and you went to middle school with casey and you have video of him falling down the stairs, please send IT to heart for and you know what I mean.

I was I didn't actually like, roll down an entire flight of stairs like a cartoon.

Let's review the footage and see what happened. Case one concern that you've brought up a couple times on the show is about what happens to teens from more marginalized groups if some of these changes to social media actually go through and whether, for example, a gay or a trans, a teenager, Young son who wants to serve, express themselves on the internet in a way that maybe their parents don't approve of how something like parental consent or notification would affect them. So do you think these new changes on instagram pose any risk to certain types of teens who may not want their parents to have to click through a bunch of approval screens to let them keep using instagram?

I think there is definitely rest of that, and we're going to have to see how that plays out. My sense is that teens still be able to explore topics like L G B T issues even in this world. And right, because people again, L G B T teens, for example, will be able to make their accounts uh, public.

For example, they want to share about their experience and if they have a supportive parent and maybe roki u doesn't have a supportive parent, but hopefully you will be able to go on instagram reals if you want to and you know search for other L G B T S like yourself and and maybe see some of that. So I think that, that is actually probably going to be okay. And honestly, like probably strikes a nice baLance.

One of the other changes that instagram is making here is after getting a lot of criticism about showing kids content related to eating disorders, for example, or other stuff that sort of might drive them to dark places, instagram is now gone to go to teens and say, here are some like basically safe topics that we can show you like you want to see cats will show you all the cats that you want. Um i'm very curious if they would put L G B T issues in that list of thirty years. So things that they pick up, my assumption would be that they would. But as long as kids can still kind of search and find what they're looking for, my hope is that um you know social media will still be there to make them feel little bit less alone.

You think this is going to a be meaningful, like you think this is actually going to make team safer on the internet?

I think in in one particular way, yes, which is this issue of strangers contacting kids online and getting them to make you mistakes is very real. There were so many investigations this year into these uh sex torture schemes are financially motivated um where you you have um these gangs of criminals who will contact random teenagers, trick them into sharing nudes and then black male them.

This has been linked to at least twenty suicides and huge misery all around to the country. This really does shut that down, I think in in some very important ways not to say that I could never happen again, but IT will be very difficult to do on instagram now. So I think that, that is a really good thing.

Now is the mental health of teenagers going to improve miraculously over the next a year as all of a sudden other instagram accounts are are made private? No, I don't think so. yeah. What do you think about some of the other changes like sending them a notification after they've been using IT for hour or halting .

notifications when they supposed to be left? I think that's all teenagers out of instagram h during the hours that they supposed to be sleeting got to stop sending them notifications. But there may be ways around that too. I think have to see teenagers are very clever, historical at getting around restrictions imposed on their their internet use, whether is by their parents or by the the social media company itself. So I think this this can be fascine to see whether teenagers are actually impacted enough by this for to matter and whether they'll surfy some clever work .

rounds yeah i'm sure some of them well, but private bite of fall is a pretty big step forward in terms of reducing the visibility of Young people on a platform. And I do wonder whether meta will be able to get some good, good will from the lawmakers and the regulators that are so mad than right now. Yeah, our regulators .

are happy about these changes. Do they feel like their concerns are being addressed? Well, I haven't .

heard from a lot of, but I have heard from a bunch of advocacy organizations that work on child safety and they were all over my inbox this week basically said this is not good enough. This is pure pr right? A lot of these advocate organizations are pushing for, uh, what they call safety by design or a duty of care.

They want to see IT in the law that this should not be a sort of industry LED volunteer project, but that legally, they have to put kids into the strictest messaging settings by default and they have to restrict the way our personal data is used. So and we've been, by the way, I think there is something to that, right? I think that this is important ough that we publish, just not leave IT to the good, good will of these four profit corporations to decide what happens if is online.

Yeah yeah I just want to say like as a closing thought to the story, like pressure works, right? These pressure campaign from parents, from lawmakers, from you know teachers, from just concerned citizens, they actually do, in some sense, forced companies to do the right thing.

And that's why it's so important to complain online. If you're not complaining online, start. It's true.

Yeah, you can change the world. Yeah what do you mind about change? IT get sappi. He told his workers to get there and us back to the office.

This park is is supported by service. Now here's .

the truth about A I. A I is only as powerful as the platform is built into service. Now puts A I to work for people across your business, removing friction infrared ration via employees, super charging productivity for your developers, providing intelligent tools for your service agents to make customers happier, all built into a single platform you can use right now. That's why the world works with service now visit service now dot com flash A I for people overwhelmed .

by to do s meet claud your new AI assistant from anthropic whether you're crafting the perfect planning a family vacation or tackling a home improvement project claude is your go to collaborator need to IP up a quick meal plan claud recipes struggling with the fox l gu from creative writing to data analysis clod brings expert level side to your daily chAllenges is like having a brilliant friend on speedy's ready to help twenty four seven join thousands already simplifying their lives with lad how A I can transform your day disc the adventures well, here we are back in the office of in which is actually relevant to the subject .

of our next story. Yes, you've heard about amazon returns. This is a story about amazon returns to office.

That's right. And I was really interested to read the story cover because I think in ways the original return to office podcast, because when we started to show almost two years ago, we thought it's important to be in the room to do what we do every week.

That's true. I love sitting across the table for me. I'm very sad when we have to recd remotely.

We've said million times, I feel the same way. And now that seems like some tech companies are starting to feel the same way about us.

Some of their the hard for effect. So this is from announcement that came out this week from C E. O andy Jessie of amazon. He wrote a letter to amazon employees and and they posted on their website it's called strengthening our culture and teams in Cassie. When you see A A memo from the boss with a subject line like strengthening our culture and teams, you know some stuff is about to go down .

yeah I see that I think a human rights violation is about to take place.

So this message was very long. And IT was basically andy, Jesse or directives to amazon employees about how they are going to change the culture of the company, including in ways like thinning out the ranks of management, are removing unnecessary layers of bureaucracy. But the thing that really grab the headlines was when he wrote that in order to serve work Better, amazon was going to require employees to be back in the office five days a week the way they were the pandemic.

yeah. And that includes fridays, which is pride. The least popular data go to the office. So you know.

that has a sting. So obviously, this has gotten a lot of attention, not just because amazon is one of the the biggest employers in america, but because they are sort of a trans sector. The things that they do, what IT comes to, uh, you know, workplace culture, often end up a rippling out throughout the economy. And this is becoming a big issue in silicon valley. I would say these companies, which were very early to allow their employees to work remotely during the pandemic, are now saying, wait a minute, we feel like we're losing control this company, we actually need everyone back yeah.

And the reason I think that this is interesting is because I think amazon has traditionally been willing to go much further in the direction of making their employees uncomfortable to get the financial results that they want. For that reason, IT really can be a bell weather. And if amazon is willing to do this, my guess is a lot of other companies .

are going to be too right. This is not a full throttle shift from amazon for the past fifteen months. So they've had a policy of quiring employees to be back in the office three days a week. We should also say this is their corporate employees. This is not the sort, obviously, the drivers and the fulfillment center employees, we've had to be back this whole time.

They they haven't figured out a way to be able to do the deliveries remotely. Desk, yeah, be great to to do that.

But as of january second, twenty twenty five for amazon corporate employees, IT will be all office all the time. And I wanted get into what this means for amazon and the tech world at large and return to office more broadly. So I wanted to bring on friend of the show, new oran's reporter carin wise, who covers amazon and is written all about this.

I spring her in.

Carry wise.

welcome back to hard work. Happy to join you guys.

So why is amazon doing this? Why are they ordering employees back the office five days a week?

Yeah, the the in the Jesse, the C. E O said he wanted to do this because he really wanted to get the kind of the culture back act. They grew so much in the pandemic that um you had so many people who didn't started the company remotely and they started going back to office three days a week and they said they felt like they really have proved to them the benefits they expected um and so that's what kind of one of the the main reasons they articulate is, is that they felt like people weren't getting the amazon culture and the company doubled in like two years as a huge amount of new people coming in and trying to learn and kind of the in is on way and you also have the antigen I CEO now and he's trying to make his kind of vision for the company, a lot of moving parts for IT. Yeah mean.

it's so interesting because I feel like during covered when they when all the tech companies decided to allow remote work and encourage remote work and allow people to do that even full time, some companies, if they wanted to, they said that basically, they didn't think he was going to impact culture. They didn't think he was going to make their teams less creative or less collaborate or whatever. Are they sort of admitting that they were wrong about that？

They were a little reluctant initially. But we're in seattle. Here was the tip of the sphere for the U. S. So they won't remote you.

One of the first big companies in the country to go remote essentially and stayed for quite a long time. So I I wonder they're like excited about IT, but they did a lot. I mean, was kind of major work across pretty much every business line and in that window as well.

But um IT is always been a very kind of intense work culture. And they wanted back. They want the old amazon back.

I mean, a lot of the language and and justice memo was like, this isn't something new. We're just going back to what I was. I was like almost like a astoria for the old days sort of thing.

yeah. I think amazon managers thrive on seeing employees tears. And when they are deprived of that for two days a week, they start to get anxious.

No comment on that one.

really. No, I got, I got no comment on the power. Alright, fine. So Carry during the pedee c amazon goes mostly remote, at least for the a the corporate employees there. But then at what point they go back to three days a week.

about a year and a half ago, they announced going back to three days a week, there was major resistance to IT. You know, the slack channels were crazy, all that stuff. But they they ve been pretty strict about IT. I mean, they, you know, monitor badgers.

There was, over the summer, A A reported crackdown on coffee bagging was the phrase where you kind of come in, how your cup coffee get your swipe and leave so that you need a proper man of time in the office. So they're serious about IT. You know some two companies have had these Mandates and by most accounts are not that aggressively enforce like maybe if you never come in sort of thing you might hear from, but that was not how what's been an amazon. You you'd even see IT like the streets in seattle, downtown or just busy traffic is coming back. The stores and select that are all very happy about this all.

And I would imagine, even three days a week, people are are building their lives around this, right? So you, I can even imagine people might choose where exactly they want to live, like knowing that, well, okay, I only have to commute three days. I'm willing to do that. So I would imagine that going from three to five might sound like a small thing, but could really massively disrupt the lives of a lot of workers.

Yeah, exactly. I have talked to people who kind of moved for at a town your driving distance, but not like a daily commutes and they'll come in and either crash the people or get a little studio or something like that and then go back and have your four days in a chiller environment um and that's kind of gone with this I was talking to someone last night who was the moon the company because he's like monday from not a azo ian you know monday on friday IT was so easy as IT into town from the east of seattle and um and now it's like going to be traffic five days a week.

Now tech companies Carry ized you know often have a lot of perks that they shower on employees for visiting the office. Is amazon one of those companies .

case you'll be shocked to here they are not one gary is one to be uh leadership principles and is no it's unna. They actually introduced free coffee during this R T O. This returned off this period um to try to lure people back and they tried to stop IT.

And then there was another up or so my understanding last I heard there was still free coffee. But no, it's not like the google plex with the law, dr. Y and whatever other offices are, they're nice, they're new, but they're not like fancy or anything like that. There's big the face or whatever.

So current, you said that part of the reason they are making this shift, at least that andy justice is saying they're making the shift to five days a week required in the office, is because they found when they brought people back three days a week that there were certain things that just we're observer ly Better. Do you have any idea what that is like where they shipping more products, where they know where employees like writing more code, like what what are they looking at to say like this is working. And so we want to go back to the office full time.

You know that he's only spoke about a very high level. This has actually been the critique from employees as like where a data driven company, where is the data behind this? I'm not saying there isn't any is just not what has like there wasn't any in the email that now this week um IT was a lot about, you know getting an understanding and feeling in the culture mattering Younger employees, the spontaneous conversations that happened in the hall, that kind of general high level culture stuff. But there was not like we found that you implemented x more code or whatever sort of thing that was not a not in the army member.

And IT doesn't seem like it's probably coming out of financial pressure either. I mean, the stock is up some like thirty four percent over the past year, like it's not like this is a company that is declining, right? So well, what are the of investor pressures here?

Yeah, I think they have jc has been focusing a lie on profit in the past couple years and has improved the profit of the company pretty dramatically. A lot of that king from the Operations though from like the warehousing part of the business and reorganizing that and making a kind of faster and actually more efficient.

Um but he's definitely laid out like at the top of this email, IT was actually talking about profit and looking for more, more space to squoze out margin. And this is kind of part of the framing is there's more work to do. They have a lot of spending up ahead, you know, for the boys are doing like A I, we talked a lot, but there's like billions and billions of dollars going into building A I and investments and data centers and working acquiring talent for that.

So there's a lot of moving pieces and they have some other big investments kind of coming online, uh, coming up that are going to expensive, like the satellites tes, that can people start a linking stuff. So there's other major things that it's not like it's mony's flowing. They have a very they have now a very profit focused mentality that was not really there in the past. They kind of shows they can turn IT and investors want more .

essentially, I I do you know. But at the same time, maza made thirty six point nine billion dollars in profit in twenty twenty three, which is, we can say, more than most companies. And so the idea that at that level of profit, the company is still worried that its employees aren't doing enough things and is going to drag them back to the office five days a week and then is going to make them go through here just to get a free cup of coffee when they walk through the door, and just really makes me worried about like every other third company in america. I always feel like, you know, if the company that made almost thirty seven billion dollars profit last year is going to give the employees free coffee, like what hope to the rest of us have?

Maybe that's why they made thirty seven billion dollars. Does a lot add up, casey.

They have so it's always .

been so it's always have a very like practical thing like you don't like a go. I don't you don't mean like there's always been that attitude know they theyve improved that with some benefits and stuff like that for sure.

They're known for not cotta their employees at some of .

the other that the work itself would be the reward essentially that is chAllenging and you exciting to work on big things.

How are amazon employees reacting to this news?

Not so well. I would say there's been a pretty overwhelming flood of concern about IT upset. You know, this is so discouraging. How does this fit with our stated goal or leadership principle of striving to be root pest employer? There is a lot of gallows humor in the lack messages i've seen, you know the means and stuck like that.

So when they said they wanted to be the the earth best prayer, I thought that was a joke. I mean, seriously.

no, that is a real leadership principle on their list along with you know frugality. And yeah.

I think a lot of employees at companies that have gone back to the office have been faced with the choice of do I do I go back to the office or do I leave and get another job from your reporting. Do you think that a lot of amazon employees are considering leaving rather than going back five days a week? Or are they mostly going to to get their teeth and put up with that?

I mean, people leave jobs over this for sure. Um so I don't think that's like a hypothetical, but you do still need a job on the other end of IT. So function is I think the question is like, are you in a role or a position that you could get another job? And for some people, probably yes. And for some people, probably not.

Yeah, I think that's one of this sort of big theories about all these returned office Mandates, tes, is that they are essentially a way to do layoff s without doing layoff s because you sort of assume that some percentage of your employees are not going to be like this or maybe they live far away from headquarters and so they literally can't do IT, and so they'll just leave. And that's a way to to reduce your headcount without actually having to .

go through the process of laying people off cheaper.

Don't have packages of is there indications produce its headcount and this is sort .

of one way to do that. Well, um this was couple with another announcement. Can I if I can go into this now because that kind of implies that potentially they are are looking to at least comfortable reducing their total headcount because they also said they want to have essentially fewer managers, that they become two periods.

Rather that you have managers propping managers for managers, for managers, for managers until they want to increase the number of employees reporting to any given manager. And that according to like the internal, if they cues on that, that means that you might be that some rules are determined, unnecessary, which IT would be losing that job. And if you can't find another job, I don't want to accept in an individual contributor role, that would be a lay off essentially.

So there's clearly some comfort without being an outcome. I can't say right now that's like the goal of IT. Certainly some employees think IT is, but that on the table I would say .

um what what we're talking about here today, Kevin, it's giving me that that funny old feeling of founder mode, right? So you know, we talked about this recently on the show. There's an idea out there that you sort of just articulated care in, which is too many of these tech companies have higher too many managers. There have been too much delegating going on and what we really need is strong centralized control, cracking the whip, getting these employees working their absolute hardest. So do we feel like that kind of a silicon valley sentiment has made its way up to seattle now?

Um I would say it's less about cracking the whip, about them being feeling bog down by a which I hear from employees all the time that it's become a very bureaucratic place like I actually think there's a lime between what justice said and what here constantly.

But they're saying like the bureaucracy slows us down and don't forget we just are in this A I moment where in zon IT was kind of widely seen as being a little late to the party, though rapidly removing really fast now. But there is this idea of like you can't move as fast as we used to and they want people to have autonomy. They talk about very amazonian.

Uh, the two pizz team. You don't want teams that are bigger than two pizza. People can can just execute on stuff. So they're trying to get .

back to some of these teams can be fed by two pizza now to be physically bigger than .

there's no discrimination case.

yes. Yes, I did here that they are announcing that they're adding thirty five hundred phone booth to the office.

Yes, we had that from the internal of accused as well. So you know we're commuting into that in, but yeah, there's little cups because you can't people got used to be gable to like have a phone call that was somewhat private or not have noise around you to squad in a place and work in some quiet. So they're working to have more meeting spaces available. They're returning to assign desks and places that had them before. And then, yes, working to procure thirty five hundred phone booths was what the internal dog said.

I think it's interesting in that one of the reasons why we saw so many companies go to remote, in addition to just the sort of obvious necessity of the pandemic, was, particularly as the pandemic started to win, there was just competitive pressure to do this right? There was this thought of, well, if we have some really strict return to office policy, we might lose a lot of our best people.

To me, I think what is most interesting about this is the sense that amazon, or just truly is not afraid to lose anybody over this, right? You know, maybe there might be some superstars that are able to work out a special deal, but I guesses those are onna. Be very few in far between if they exist at all. And amazon's is not worried about people going across the street and working at a microsoft or maybe the the meta offers up in seattle. So why do we think that is .

IT does show a confidence in in their position in the draw market. I would say a lot of these are implemented during the great reshuffle, whatever being called, where people were changing jobs a lot. This was one way to accommodate that into the Laural people in.

But there shows the confidence that mean there kind of amazing, like amazon has grown a lot in the past several years and their headcount is basically flat, like it's if you you know, and that includes their early employees, but it's it's essentially flat. So they are doing more with less. And I think there's a the the macro environment changing, and they're clearly nervous about that. They talk on the learning calls about like just the uncertainty at the moment right now.

So current, I wanted talk about another thing that I learned about from your reporting, uh, which is A A what's called being called the bureaucracy mailbox. This is something that andy, Jessie has recently set up. And it's a place where amazon employees can send any examples of bureaucracy, unnecessary red tape, uh, that they can sort of make. They can cut out and run more efficiently. What do you think the input is for that .

was IT that seems sense of like bloat and just loners and frustration. I think that he he's not wrong in like I people tell me this all the time. There's a fear amazon of being day to and a lot people think there are there and they should just embrace that and like acknowledge IT and me on.

but that's .

not always day one. And amazon, which means you're always building from scratch. You have that stuff not start up. Scrappy mentality 的 一种。 Rest on your law, als. You always have to pretend, not pretend, but believe that nothing is a given and you can lose any advantage you have any day.

And it's like very, very there's literally like building in the headquarters headquarters building at the headquarters here is called day one like it's a big concept um so there's I think a fear of the day to us. And this is kind of a the mailboxes, a symbol of IT where the people will use that. I cannot say.

yeah, I was we were actually so inspired by this a this move by andy Jesse that we recently set up a hard fork bureaucracy mailbox. And casey, have you made any submissions to the heart for bureacracy?

I did bring something I would like to submit to the bureaucrat.

Kevin.

You know, we have a podcast and something that podcast sometimes you make. And when we have had conversations about how to make merchandise, what i've been told is the number of meetings that would have to be held at the new times to make merchandise would be so great that um IT would not be worth IT. And I don't know about you, Kevin, but I would love a beautiful hoody, maybe a crown and settle a cape, just something tasty that I can wear into society and let people know that i'm affiliated with the hard for podcast. And so I would love to see the times work on that so that we could have some beautiful merchant dice for ourselves .

and for all the fans. Okay, well, let's submit that to the mail box. How about you? Well, I I have not been anything, but I did find an anonymous tip that was left in the hard fork bureaucracy mailbox.

And I was all just read out loud here as dear hard fork bureaucracy mailbox i'm writing to complain about an inefficiency. We have to read the credits every week, even when they're exactly the same as last week. This makes no sense.

They should just copy and pace the ones from last week. No one would notice. I want my forty five seconds.

No, I don't know who sent that, but clearly they're very angry. Oh, you know, IT goes on. IT says. I'm also writing to complain about the fact that despite the veneration in our podcast studio being fixed recently, so it's not literally a thousand degrees here all the time, the studio still lacks basic amenity such as well decor, working ipad monitors and a button Kevin can pushed to administer a small electrical shock to casey every time he starts talking about productivity apps.

So I actually was to support that reaux racy mailbox complaint right up until the end there.

Yeah, that goes on. It's still not done. And while i'm here, I also wanted complain about the kansas city marry out that almost prevented from us from taping a show a few months ago.

The wifi is horrible even after Kevin up greater to the premium inversion. I assume this is due to stifling bureaucracy in the marriage corporation or possibly the workmen virus. Please fix IT.

But is that for the heart, for reactors.

acy and box with the marriage? Well, people are very confused out here. They're very angry.

and I don't know who said this one. So lad s.

yeah, yeah, I whoever sent that and I you really speak my mind. Yeah, yeah, yeah.

I IT is so funny to me that this has to be a separate in box is presumably ed Jesse already has an email address. He could just ask his assistant to pull off the bureaucracy related complaints. But yeah, maybe there is something special about setting .

up a dedicated in box intention .

intention much .

to join you guys.

You're taking a late afternoon grow from outfits to sports highlights to wedding picks and new york times cooking post stops you in your tracks. The most delectable regardest she've ever seen. Sadly, life gets in the way the grocery lines, the cars in the shop that show is on.

But on the recipe, you see a button that changes everything. Shop ingredients on instincts, music, tear taste buds, get ingredients delivered in as fast as thirty minutes. Learn more at M Y T. Cooking up com slash indecent this .

page as is supported by made in cook ware as a .

chef and a restaurant honor. I'm as meticulous about my coward as I am about my ingredients. That's why I love made in cut war.

Each pen they make isn't just designed to perform. It's crafted to last. As a mom, I love that I can trust me in.

It's made from the world's finest materials so I can feel good about what i'm feeding my family. I'm chef rick Williamson, and I used made in coker for full details. Visit made in coquart dot com. That's M A D E I N coquart dot com.

Hard focus produced by witney Jones and Rachel coon, were edited by gene poet, or factor ked by kate in love. Today's show was engineered by Daniel romeos original music by Elizabeth youtube, maria lizana, roman emos lia shot damron and then power our auditor senator is not globe video production by ryan manning and Chris. You can watch the full episode on youtube.

At youtube 点 com splash hard for special thanks to policeman wing time， dalia hadad and jeffrey mora. You can email us, as always, at hard fork at N Y times that come or you can send a tip to the hard for bureaucracy mailbox at casey newton at platformer to use. I don't actually know that through your email address.

This page as is supported by made in coke ware.

As a chef and a restaurant owner, i'm as meticulous about my cock as I am about my ingredients. That's why I love medium cut wear. Each pan I make isn't just designed to perform.

It's crafted to last as a mom, I love that I can trust me in. It's made from the world's finest materials so I can feel good about what i'm feeding my family. I'm chef rk willison, and I used made in cocoa for full details. Visit made in a dot com that's M A D E I N coquart dot com.