cover of episode Infosec & OpenTelemetry

Infosec & OpenTelemetry

2024/11/1
logo of podcast Ship It! Cloud, SRE, Platform Engineering

Ship It! Cloud, SRE, Platform Engineering

People
A
Austin Parker
A
Avthar Suwathan
D
Dave Rosenthal
K
Kurt Mackie
Topics
Dave Rosenthal: 讨论了 Sentry 如何将各种来源的遥测数据(日志、指标、错误)通过 trace ID 关联起来,从而实现更全面的应用健康监控和更便捷的调试体验。他强调了将所有遥测数据关联起来的重要性,这使得开发者能够更深入地分析数据,并更容易地找到问题的根源。 Kurt Mackie: 他批评了公共云的不足之处,认为它们的设计目标并非为了提高开发者的生产力,而是为了平台团队。他介绍了 Fly.io,一个为开发者设计的公共云平台,旨在简化应用部署和管理流程,并提高开发效率。他认为,如果能将复杂的问题简化为更短时间内可解决的问题,开发者就能构建更多有趣的应用。 Austin Parker: 他认为安全策略和控制是安全方面最大的障碍,过多的安全控制会增加摩擦,导致人们绕过这些控制。他介绍了 OpenTelemetry,一个用于标准化可观察性数据的项目,旨在解决数据孤岛问题。他强调了标准化数据的重要性,这使得不同团队能够更好地协作,并更容易地解决问题。他还讨论了 OpenTelemetry 的未来发展方向,以及如何提高其易用性。他认为,OpenTelemetry 的目标是让可观察性数据成为内置功能,并支持应用程序和系统级别的遥测数据。 Avthar Suwathan: 他介绍了 Timescale,一个基于 Postgres 的公司,构建云端和开源工具,帮助开发者更好地使用 Postgres 进行时间序列分析和 AI 应用开发。他强调了使用 Postgres 构建 AI 应用的便捷性,以及 PGAI 和 PG Vector Scale 项目的开源特性。 Justin Garrison & Autumn Nash: 两位主持人主要围绕着家庭生活与软件开发的类比展开讨论,例如将孩子的行为比作软件开发中的问题,以及如何利用项目管理和自动化来解决这些问题。他们还讨论了儿童对技术的依赖以及由此带来的挑战。

Deep Dive

Key Insights

Why are public clouds often perceived as difficult to work with for developers?

Public clouds were designed for platform teams to build infrastructure, not for developers to deploy apps easily. They often require significant upfront effort and lack intuitive developer-focused tools, making it harder for developers to be productive.

What is the main goal of OpenTelemetry in observability?

OpenTelemetry aims to unify observability data by providing a standard way to structure telemetry data, instrumentation APIs, and semantic conventions. It allows developers to emit consistent data across different systems, making it easier to analyze and understand systems without worrying about different logging or metric formats.

Why do security policies often become obstacles to productivity?

Security policies, such as password rotation or device lockdowns, are often designed without considering how they impact users' ability to get work done. This friction leads users to bypass controls, undermining the intended security benefits.

How does OpenTelemetry handle data transformation for different use cases?

OpenTelemetry supports lossless transformations, allowing metrics to be translated into different forms (e.g., delta, aggregation) without losing data integrity. This flexibility enables different teams (e.g., security, app dev) to process and analyze data in ways that suit their needs.

What is the biggest challenge in building OpenTelemetry?

The biggest challenge is governance and community management. With thousands of contributors across multiple languages and tools, keeping everyone aligned and motivated while balancing cross-company demands is complex.

How does OpenTelemetry integrate system-level and application-level telemetry?

OpenTelemetry uses the collector tool to gather data from various sources, including system logs and metrics. It can associate application-level telemetry with system-level data using trace IDs or resource attributes, allowing for unified analysis across both levels.

Why is OpenTelemetry considered low-level and dense?

OpenTelemetry provides highly structured, low-level data that is rich in detail but can be overwhelming. It requires tools to process and transform this data into usable formats, which can be a challenge for users who prefer more high-level, out-of-the-box solutions.

What is the future vision for the internet according to Austin Parker?

Austin envisions a return to an internet where individuals own their digital spaces, using tools like AT Protocol to control their content and identity. This decentralized approach contrasts with the current model of renting space from large corporations.

How does Timescale enable AI development with Postgres?

Timescale provides open-source tools like PGAI and PG Vector Scale, allowing developers to build AI applications (e.g., RAG, search) using Postgres and SQL. This eliminates the need to learn new technologies, making AI development more accessible.

Why do developers often bypass security controls in organizations?

Developers bypass security controls because these controls often add friction to their workflow, making it harder to get work done. When security policies don't align with how work is actually done, users resort to workarounds to meet their goals.

Chapters
This chapter explores the humorous challenges of parenting in the digital age, particularly the unexpected ways children interact with technology and the temptation to apply project management tools to family life. It includes anecdotes about kids' interactions with smart home devices and online content, highlighting the surprising levels of technical aptitude and resourcefulness in young children.
  • Parents use technology to manage kids' activities.
  • Kids quickly learn and adapt to new technologies.
  • The line between project management and parenting is blurred.

Shownotes Transcript

Translations:
中文

This is Ship It with Justin Garrison and Autumn Nash, a podcast about everything that happens after Get Pushed. Ship It is brought to you by Fly.io, the home of changelog.com. Launch your apps close to your users all around the world. Learn how at fly.io. ♪

What's up, friends? I'm here with Dave Rosenthal, CTO of Sentry. So Dave, when I look at Sentry, I see you driving towards full application health, error monitoring where things began, session replay, being able to replay a view of the interface a user had going on when they experienced an issue with full tracing, full data, the advancements you're making with tracing and profiling, cron monitoring, code coverage, user feedback, and so on.

and just tons of integrations. Give me a glimpse into the inevitable future. What are you driving towards? Yeah, one of the things that we're seeing is that in the past, people had separate systems where they had like logs on servers, written files. They were maybe sending some metrics to Datadog or something like that or some other system. They were monitoring for errors with some product, maybe it was Sentry. But more and more what we see is people want all of these sources of telemetry logically tied together somehow.

And that's really what we're pursuing at Sentry now. We have this concept of a trace ID, which is kind of a key that ties together all of the pieces of data that are associated with the user action. So if a user loads a web page, we want to tie together all the server requests that happened, any errors that happened, any metrics that were collected. And what that allows on the back end

You don't just have to look at like three different graphs and sort of line them up in time and, you know, try to draw your own conclusions. You can actually like analyze and slice and dice the data and say, hey, what did this metric look like for people with this operating system versus this metric look like for people with this operating system and actually get into those details. So this kind of idea of tying all of the telemetry data together using this concept of a trace ID or basically some key, I think is interesting.

is a big win for developers trying to diagnose and debug real world systems in something that is we're kind of charge the path for that for everybody. Okay. Let's see you get there. Let's see you get there tomorrow. Yeah, perfectly. How will systems be different? How will teams be different as a result?

Yeah, I mean, I guess again, I just keep saying it maybe, but I think it kind of goes back to this debug ability experience. When you are digging into an issue, you know, having a sort of a richer data model that's, you know, your logs are structured. They're sort of this hierarchical structure with spans. And not only is it just the spans that are structured, they're tied to errors, they're tied to other things. So when you have the data model that's kind of interconnected, it

opens up all different kinds of analysis that were just kind of either very manual before, kind of guessing that maybe this log was, you know, happened at the same time as this other thing, or were just impossible. We get excited not only about the new kinds of issues that we can detect with that interconnected data model, but also just for every issue that we do detect, how easy it is to get to the bottom of.

I love it. Okay, so they mean it when they say code breaks. Fix it faster with Sentry. More than 100,000 growing teams use Sentry to find problems fast, and you can too. Learn more at Sentry.io. That's S-E-N-T-R-Y.io. And use our code CHANGELOG. Get $100 off the team plan. That's almost four months free for you to try out Sentry. Once again, Sentry.io.

Hello and welcome to Ship It. I'm your host, Justin Garrison, and with me as always is Autumn Nash. See, we ended up going like two hours yesterday and now we're like, we started before the intro. I think we're just going to roll most of that intro right now. And it's a whole conversation about kids learning how to project manage and automate.

My daughter is like absolutely smitten with our smart, you know, turn the lights on and off our voice and she will run up to like,

When Mandy, my wife, is on the FaceTime with someone, she'll run up and yell, like, turn off the lights to try to turn their lights off on the other side of the phone. Because she has an aunt that will do it for her. When she comes up, she'll... And pretend. Well, no, she'll actually have Alexa lights, so when Dylan runs up and says it, she knows. She'll say, Alexa, turn off the lights, and the lights will turn off, and Dylan's just like, yeah.

over the phone that's great yeah because she hasn't figured out like she keeps trying but siri is like siri she's just not good enough at the words yet for siri to pick up on it yeah

Like she's trying, but Siri just does not. Life gets real when like they can understand your children because all of a sudden you have six things in your Amazon cart that you don't know where they come from. I'm not, I'm not really looking forward to that moment of the addiction getting good. Don't let them find out that there's a fart song and a bunch of other stuff because like. Oh my God. Like.

Yeah, I'm probably going to have to rethink my opinion about smart homes as soon as in another year or so. Yeah, my kids would for like an hour spend just like changing the color of all the lights. They're like, wait, make them blue. Oh, now make them orange. Somebody put the longest song ever or what is it? The song never ends. Yes, on Alexa. Why though? Why? Yeah.

I was like, do you just hate us parents? Like what is wrong with you? I just think about like how much money like I have YouTube. Like I got YouTube premium because we will favor YouTube kids and just like it's limited to like certain channels. I'm just thinking like how much money is Miss Rachel made off of me? Oh, Miss Rachel is balling. Forget that. How much money has Ryan made off of all of us? That kid's got $21 million and he's like seven. Yeah.

the economics of of children's youtube i i don't like to think about it because i just try to ban my kids from most youtube because i'm tired of watching it and it makes me mad that they really believe half of it and i'm like that's fake yeah i just it's like it's i mean she's three like my daughter's three so it's she's still you know pretty young and it's

It's really just like, okay, please watch the tablet while... Yeah, you try to survive. We're in survival mode, so it's kind of like, okay, you get Miss Rachel, you get PBS, and you get Super Simple Songs. Go over here while we're trying to do something. That used to be us and Blippi, but thank God Blippi now is on Prime. We never got Blippi, thankfully. Don't do it.

It's so creepy. My youngest wants Danny Go right now. Oh, yeah. I'm hoping that we can stick to the...

Normal or the not weird corner of children's YouTube. It's so weird. Yeah, it'll probably last until she gets older and then her wonderful little friends at school will tell her about all these fun things she doesn't get. It's like my youngest watches. They watch like a Danny Go YouTube video in class. And so he comes home and he's like, I have to watch Danny Go. I'm like, no. Great. Thanks. Thank you. Educational system. I've banned them to like Epic, but they even watch like YouTubes and like

and like different like Doug, like Doug and all these different questions links, which I mean like they're schooled at, they're like educational. But I'm like, if you can't find it on Epic or like PBS, like you're out of luck. They learned how to Google though and ask Alexa how to do stuff. Oh.

And they found out so many ways around system child locks that at this point, they should hire my four-year-old to work for the damn NSA. He figured out PlayStation's entire child lock thing. And tell me why he can't play Plants vs. Zombies. And he can't play regular games, but he's figured out how to re-download Fortnite six times. Like...

Who made that? The amazing thing about children is the level of motivation for highly specific tasks. Yes, I can't get them to clean their rooms, but let me tell you, they found a way to get on Google and Amazon through my child protection locks. And they're like, Mom, this is the Halloween outfit costume I want. And then here's this thing. And I'm like, how did you? First you're like, no, not that. And then you're like, wait, but how did you get there? No.

Like it hits you. Yeah. It reminds me of that bit that went around a few years ago where it's like some girl was like, Oh, I got banned. Like my parents took my phone. So I'm tweeting from my fridge. And I have the Twitter fridge. That's some shit my kids would figure out. Like, like,

I just sat there for a minute and I thought about how many screens are in my kitchen and living room because it's one big open room. And it's like, at one point my kids were watching YouTube on the Alexa show. Then they were watching it on the fridge and then on the TV. And I was like, you can see each screen from each wall. Like, what are we doing? Oh, God. The worst is when they watch different things. Oh, my God. My kid does that. He'll be watching TV live.

Yep, she'll have PBS on the TV, and then she'll go over and get the tablet and watch Miss Rachel on the tablet, and then do one of these videos.

Just watch the TV and watch the tablet at the same time. My oldest is playing Minecraft, watching a YouTube video and on his tablet. And I'm like, what are you doing? Amazing level of multitasking. Okay, my kid will build a game in Scratch, right? While he's watching TV, making his brother play one game that he made as his beta testers that he's yelling at on his laptop. And I'm just like, you can't yell at your testers, bro. Like, what are you doing? Like...

Your brother's going to unionize. I swear. I hope they do. I hope they're like, we want one piece of candy per game testing. Collective voice now. Yeah. Collective bargaining. This is never too early to teach labor relations. I'm telling you. I was like, first of all, those are your beta testers. And then he was like, they were asking for different things and to change it. And he was like, that's not how you play it. And I was like, then you didn't write it clear enough. Right.

Welcome to users. He was so mad. And he was yelling at him. Yes. And I was like, that's not how this works. You should just use this as like the cold open. This is. The entire conversation just needs to be like, just drop it in at the beginning. People are like, what show is this? Yeah.

Welcome to Ship It. The podcast is all about your kids learning how to manage software. Learning how to teach your kids about project management. It's never too early. Okay, but hear me out. We should do one where we just have toddlers sitting here talking about...

like tech and Minecraft, it would be so funny. Like, like it'll be like the faces of ship it, but like mini shipping, we'll put like the headphones. You get, you get one of them to have a scratch app and all of them talking about like how they're using it wrong. Kid gets mad downloads on scratch. Okay. Like he gets mad downloads. Like he is a podcast of eight year olds talking about Minecraft mods would probably be like utterly fascinating. It would also,

would also get our kids off our backs. Okay, at one point when we all worked at the same company, we were like, okay, if we get one Minecraft server, we can get through the summer. Why didn't you make the server, Justin? Because Logan doesn't care anymore. He just went on a friend's server. Logan's a grown man, okay? What did you do? He has his first dance tonight. What? He is going to a dance. I know. He sent me a picture of Logan the other day and I was like,

Who's who? What grown man is with you at the baseball hockey game? I'm a few years out from all that, thankfully. Oh, it happens quickly. Okay.

Like, I have a 10-year-old, a 7-year-old, and a 5-year-old. And he was like, Mom, don't help me get out of the car. My friends are watching. And I was like, whoa, what? I was like, I wipe your butt. Like, what do you mean? Our business is now. We still got just the one, and she's three. So it's... Just don't get outnumbered. Just FYI. Like... Trust me. Trust me. No...

I had the one after the first mental breakdown I had in the bathroom when she was like six months old. Like, I'm just like, all right, we're done. You're going to have more of those mental breakdowns. But it's just like, you know what? It's going to be two on one for the rest of this. But hear me out, though. I don't even got to be good at video games because the three of them will beat up on teenagers and they carry my ass in Fortnite. It's crazy.

There are advantages. I don't even carry groceries anymore. I'm just like, get the, I Instacarted it. Like I paid for it. Like get to carry me to the crew right now. Oh,

He was like, my kid was like, why don't they just like automate taking out the trash? And I was like, we did. I have, I had you. Congratulations. You are the automation. Also like it would be my kid that says we should automate taking out the trash. It's like, I broke you.

This is our 30th show, which is awesome. Thanks for, we've been doing this for just a little while now and that's been awesome. Today on the show, we have Austin Parker, who, Austin, you work at Honeycomb. I don't actually know your title. Director of Open Source. Director of Open Source. Awesome. And we're going to talk a lot about open telemetry today, which is one of the things that I was interested to hear more about from you because that is a topic that I know very little about and I always want to learn more.

But again, we went through this whole background of kids and building their own stuff and automation. And I think at the end of the day, we ended up with like, kids, you're on the screen too much, so you need to start a podcast. And that's like where we...

And we're going to solve some of this problem. Well, we also have come to the conclusion that we've created our own baby version of the nerds that we are. I think that that's your prerogative as a parent, right? Like you just want to, your prerogative as a parent is like, okay, I have all these menial household tasks. And so I'm going to invest in a, I'm going to invest in a child so that within 10 years they can take the trash out and get mad at me about it.

I asked them to, I think one of them to clean the room. You would have thought that I was murdering them. And I like, I asked my son to wear a jacket to school. It's 45 degrees outside. He was so mad that he couldn't wear shorts. And I was like, I'm sorry that I'm trying to keep you from dying of like frostbite. He was like, I'm going to get heat stroke when it goes up to 50 today. And I was like, you know, kids and users have a lot in common. Yeah.

It's like arguing with tiny terrorists. Yeah. The thing that you can, you can fire customers, but you can't fire your own children, which is, um, bad decisions were made. This was not thought out, but it reminds me of all those times you see people on Twitter or whatever. It's like, Oh, I, I, we both work in software. And so we create, we, we have a sauna or a Jira for our household. I'm just like, who hurt you? Yeah.

Okay, like, don't get me wrong. At one point, I did make my kid write, like, a one-pager on why he wanted a bearded dragon. And then I was like, oh, we've gone too far. No, that's too much. I think you've gone officially beyond the pale when you start doing, like, OKRs for your children. Which is honestly such a... I mean, it's terrible that I said it. I've birthed this into the world and at some point... Damn it, someone's going to do it now. It's your fault. It is.

Which is really the story of my life. You say something and regret it later when it happens? That's how the Deserted Island DevOps happened. I made a joke on Twitter, and then I'm the Animal Crossing DevOps conference guy. Is that still going? No. Wait, Animal Crossing DevOps? Go back! Deserted Island DevOps, the world's first...

virtual DevOps conference held in Animal Crossing. We ran three years during the pandemic. I'm going to buy a Switch right now. You can watch it on YouTube. Deserted Island DevOps on YouTube. You can watch all the talks. It was great. Were they animals? I mean, we used the multiplayer in... Oh, God. Whatever the newest one is, right? And people would bring their villagers onto the island and then I set up a little room that looked like a conference room. A little podium.

And they would give their talks and they live streamed

the switch and mucked in the audio from them talking over a Zoom and then overlay their slides. It's very cute. Go look it up on YouTube. This is the most amazing thing I've ever heard of. We'll have it in the show notes. I was just going to say these are going to be the most fire show notes ever. Yeah, desertedisland.club or on YouTube, Deserted Island DevOps. Everything should still be up there. But yeah, we ran the last one a couple of years ago.

And it was very fun. It was like the, it was an inversion where, because it was kind of people were traveling again. We, uh, we found an Island in Michigan called, uh,

They had like a hotel on it and an event space. And so all the speakers came in and the speakers were all in the same place. And then we still live streamed the Switch, but people were all in the same room together. It was a nice way to kind of put a bow on that little experiment. You should bring Animal Crossing DevOps to DevOps Days LA at scale and it would be so much fun. Like we'll all just sit there and play like Animal Crossing.

crossing. We could, you know, maybe I might come back at some point. I keep meaning to go out to scale, but the DevOps days LA is just, it's never at a great time for me. March in Pasadena though. I mean, come on, this is, it's usually like either so close to KubeCon or it's scheduled against KubeCon EU that I, I,

Like the travel doesn't work out. Yeah. Especially if you're traveling twice for two different conferences in different directions, it gets difficult. Yeah. Especially Europe and then back, you know, West Coast. I live on the East Coast. The whole time I was like, take me to Paris with you because everybody was going to cucumber. I was like, put me in your packet, like your suitcase. I just want a croissant. The croissants were really good. Dude, everybody was eating croissants on Twitter the next week. And I was like, I hate you.

Don't worry. We're in, we're in London in 2025. So much love to all my friends in the UK, but you know, British cuisine is an oxymoron. I wouldn't be as excited about the food.

Lovely people, though. I do want to go to London. I want to go see all the sights. I just went a few weeks ago and I drove a Mini Cooper, a classic Mini Cooper for the first time. Rented one. Fantastic. Did you drive on the opposite side of the road too? I did. The steering wheel was on the right side. We were driving on the right side. It's a stick, so it's manual. So I had to shift with my left hand for the first time. And I did okay. I think I got my first speeding ticket.

I saw a flash of lights as I was going off to return. I know. I was like, wait, how are we not sure? They have a lot of automated speeding ticket cameras throughout. They love their automated cameras over there. Yes. And so I was going to return the car and I went a little faster on what I thought was a highway, but it turns out the speed limit was not highway speeds. So I saw a flash of light and I'm still waiting to see if the ticket comes in the mail. You went to London and this is the story that you came back with? Yeah.

You're like, I went and I got my first. If they get it across the pond to you, then that's impressive. Yeah.

I mean, it's licensed under the rental company's name. So I told them, I was like, hey, by the way, I may have got a ticket. I saw a flash of light. If you get it, email me. I was like, if they have a picture, I want it. Yeah, I want it. I want to frame it. I want to put it on my wall. It'll be framed. Justin's the only person who will be like, hey, I could get out of this speeding ticket, but I want the picture. So like, here's my contact information. Please come after me because I just want the picture of it. My first speeding ticket. It's fine. It's great. I drove on the wrong side of the road and all I got was this...

speeding ticket this is also the man that has pictures of his broken arm on like the internet so like just why did you put it somewhere what what else is the internet for i don't know hey friends you know we're big fans of fly.io and i'm here with kurt mackie co-founder and ceo of fly kurt we've had some conversations and i've heard you say that public clouds suck

What is your personal lens into public clouds sucking and how does fly not suck? All right. So public clouds suck. I actually think most ways of hosting stuff on the internet sucks. And I have a lot of theories about why this is, but it almost doesn't matter. The reality is if like I've built a new app for like generating sandwich recipes, because my family's just into specific types of sandwiches that use Braunschweiger as a component, for example,

And then I want to like put that somewhere. You go to AWS and it's harder than just going and getting like a dedicated server from Headster. It's like, it's actually like more complicated to figure out how to deploy my dumb sandwich app on top of AWS because it's not built for me as a developer to be productive with. It's built for other people. It's built for platform teams to kind of build the infrastructure of their dreams and hopefully create a new UX that's useful for the developers that they work with.

And again, I feel like every time I talk about this, it's like I'm just too impatient. I don't particularly want to go figure so many things out purely to put my Sandwich app in front of people. And I don't particularly want to have to go talk to a platform team once my Sandwich app becomes a huge startup and IPOs and I have to do a deploy. I kind of feel like all that stuff should just work for me without me having to go ask permission or talk to anyone else.

And so this is a lot of, it's informed a lot of how we've built Fly. Like we're still a public cloud. We still have a lot of very similar low-level primitives as the bigger guys. But in general, they're designed to be used directly by developers. They're not built for a platform team to kind of cobble together. They're designed to be

useful quickly for developers. One of the ways we've thought about this is if you can turn a very difficult problem into a two hour problem, people will build much more interesting types of apps. And so this is why we've done things like made it easy to run an app multi-region. Most companies don't run multi-region apps on public clouds because it's

functionally impossible to do without a huge amount of upfront effort. It's why we've made things like the virtual machine primitives behind just a simple API. Most people don't do like code sandboxing or their own virtualization because it's just not really easy. There's no path to that on top of the clouds.

So in general, like I feel like and it's not really fair of me to say public cloud suck because they were built for a different time. If you build one of these things starting in 2007, the world's very different than it is right now. And so a lot of what I'm saying, I think, is that public clouds are kind of old and there's a new version of public clouds that we should all be building on top of that are definitely gonna make me as a developer much happier than I was like five or six years ago when I was kind of stuck in this quagmire.

So AWS was built for a different era, a different cloud era and Fly, a public cloud, yes, but a public cloud built for developers who ship. That's the difference. And we here at ChangeLog are developers who ship. So you should trust us, try out Fly.

Fly.io. Over 3 million apps, that includes us, have launched on Fly. They leverage the global anti-cast load balancing, the zero config private networking, hardware isolation, instant wire guard VPN connections with push button deployments, scaling to thousands of instances. This is the cloud you want. Check it out. Fly.io. Again, Fly.io.

Austin, I think we started this conversation on Blue Sky as you were ranting about security. Yes. And it was a while ago, and I don't even remember what the context was, but I remember it was enough to say, you want to come on the show and talk about it. Yeah, I think my actual rant was something around the biggest obstacle to actual security is actually just security policy and controls.

It's super interesting to me because if you think about, to extend the metaphor a little bit, right? Or by way of example, password policies. Everyone knows like, oh, you have the password. It's like, you can't use the same password twice. You can't use the same, you know, it can't have your date of birth in it. It can't have your name in it. It can't have all this stuff in it. And for years, we just all kind of like accepted like,

okay, my password is going to be this one random string I memorize. And then every month I add another, I increment something at the end or I add a number to it or whatever. And it doesn't actually increase security because I'm using the same password everywhere.

And so, you know, slowly over time, the standards change. And now NIST says like, oh, you don't actually have to do like password rotation every month for users is not recommended. Right. But I think you see this in so many areas of like security, especially, you know, you give someone it's like, oh, we want to make sure that people have like super locked down corporate devices. So we put, you know,

all these agents and we have all of these timeouts and dah, dah, dah, dah. And what do people do is like, they want to get stuff done. They see the security controls as an obstacle to getting stuff done. So they just completely bypass the security controls. They use personal devices. They use shadow IT, dah, dah, dah, dah, dah.

I'm not saying that those people should do that, right? Like, I don't work in security. And you should never take my advice about it. But I think it's actually super instructive to the rest of how we think about software systems and systems in general is...

There's so many best practices and things that we do and say, well, you just do it this way. Just do this thing and that's going to solve your problem. But the more friction that those things add to the experience of whatever you're trying to actually accomplish, then the more people will cut around them in order to make it easier for themselves to do what they actually want to do or need to do. And it's not just security, right? It happens in observability. It happens in CICD. It happens in DevOps. It happens in

anything you want to point to, the more controls you put in a way, if you're not thinking about how do these actually fit into how work gets done, then you're going to run into people bypassing the controls. That's so true. Like going from a solutions architect to like, actually like being a developer in production, it is wild. Like when you're trying to give people best practices and then seeing how things work in production and how like,

You want to do best practices, but at the same time, there are so many things that don't work the way that best practices and architectures are usually diagrammed. I almost wish people had more thought into that because they make it really hard when they kind of like imagine these controls or these things.

I don't know how you'd explain it, but these different advice. What the ideal world would look like if we could implement this thing. Yeah, it's almost doing it in a world that doesn't actually exist. And I'm like, can we get something that's a little bit more reality-based so that way we can actually follow those rules? I mean, a lot of it is just like intro to physics stuff, right? Like when you think, you know, go back to high school or college and you think about like physics education where a lot of it's like, okay,

okay, assume, you know, a, a frictionless plane and then do this math or this formula. But it's like, okay,

Okay, but in the real world, I can't assume that. And the reason that we say, like, assume we're frictionless, when we're teaching people, when we're educating, I think a lot of times we do, we want to take that complexity or we want to take these extrinsic forces and sort of say, like, okay, set that aside for a moment. And the hope is that by taking that part away, by taking all that complexity away, we can get at what's, like, really important, like velocity, like calculating velocity or these other, like, really core foundational concepts.

The problem is that in software, especially, I feel like we kind of, we miss that second part, which is like the re like, why do we assume a frictionless plane? We assume a frictionless plane. Cause if you don't get that higher level, understand if you don't get like the, the friction, the plane is not actually important to the concept you're trying to get through, like, you know, how to calculate the velocity or, you know, decay or whatever based on mass. Yeah.

And if you don't understand that concept, then you're never going to get to the point where adding in the friction and adding in hills and all that other stuff is going to actually make a difference. But then we go into like the software world and we start out by assuming like everything must be a frictionless plane. It must be a perfectly spherical cow.

security, observability, logging, deployment, whatever you want to point at, we package all of our lessons up in this way that is completely ignorant of the context surrounding how work gets done at organizations, how different one system is from another, even systems built in the same tech stack, and what is really, what are the things that are not just driving value, but like

the decisions that lead to what gets prioritized, what drives value in not just for a customer, but what drives value in the engineering organization. What is the thing that gets you promoted or gets you a good review or hits your OKR? All of that, I think,

That's the plane. That's all the extrinsic forces bouncing up against our best practices. And unless we kind of think about that as engineers and as people working in production or people talking about how you should work in production, then we're just going to keep running into this problem of,

Telling you all the 101 stuff over and over again. And then when it doesn't work, being like, well, you didn't do enough best practices. You're doing it wrong. You should have best practiced harder. I think that's the most frustrating part about being an engineer. It's like there's so many things that will teach you like the concepts of writing code or the concepts of doing like general things. But there's so many more things.

nuances and context that you need to actually write production code at scale. And we do a horrible job at teaching people that. And just like,

giving people the documentation. And sometimes you run through documentation and you're like, did you even do this to see if it worked before you released this on the internet and told people this was going to be how people use your product? Some of the most successful databases are successful because they have good documentation and it's easy, not necessarily because it's the best tool for it. Yeah. Especially docs, they make so many assumptions about

almost all of them, they all have to focus on an individual because there's one person reading the docs. And I had this problem over and over again at Amazon where I kept telling like all of our docs were like, you should be able to go do this thing and deploy it to Kubernetes. And I'm like, wait a minute, this works for me at Amazon because I'm an admin on this AWS account.

Go to any other company of someone trying to do this and they're going to go email four people to get access to do that thing that you just told them. Oh, this will be easy. I mean, this is one step. I'm like, no, no, no. Let's break this up into like I have your admin account credentials and you have this amount of access to a cluster. Now go follow your docs and tell me where it's going to fall apart because you have to switch back and forth to an organization complexity of like now there's four teams.

One manages the database, one manages the network, one manages the cluster, one's AWS account. Now how are you doing this? None of that practice works anymore. And that's exactly also what you're talking about. Now I'm like, now I just make a friend over on that other team and say, hey, can you do this for me? I'm not going to file a ticket. I'm not going to go through all the steps. I just need to get this done. I think there's a really interesting corollary here with stuff like Open Policy Agent. And I, again, not a security person, but OPA, my understanding is it's a way to write manifests about security policies for code.

And I think there are analogs to this from various clouds. And it's actually, it's a cool idea. It's basically like, okay, here's like a manifest file that describes all the different sort of like security stuff for this service. And like, what ports should it talk on and what routes should it have? And what is the ACLs and da, da, da, da, da, da, da. But when you get down to it, it's kind of like a lot of it looks like

A bunch of security people kind of went and looked at all this code that the developers were writing and said, I don't really understand all this code. I want to take all the things that I care about. I want to put them in this little security box over here. And then I want to write tooling that interacts with the security box. And then I'm going to make all the developers figure out how to translate what they're doing into things that fit in my security box. And in a lot of ways...

I think that the legacy of the cloud transformation has been that in a lot of ways, where we try to

We build everything on top of abstractions, but we try to make the lines between those abstractions a little less gray, a little less porous than maybe they would have been in the past so that we can say like, okay, build and deployment, those are CI, CD problems, those are DevOps problems. And they go in the DevOps box and the DevOps people use this stuff. And security stuff, that all goes into the security box and dev stuff goes into the dev box. And we've

In doing so, we've kind of lost the entire lesson of Agile and DevOps to begin with, which was it's about working code over a process, right? It's about responding to needs. The entire idea of DevOps is... Reducing friction. Reduce friction. You should own your code as a developer, and you should own it on its entire journey into production and even after production. I wrote a blog about this a while ago. I think I called it the...

of DevOps. And SRE is going through this similar sort of thing where you're seeing now, like you see AI, AI-powered SREs that you just like turn all these AI agents open on your cloud account and they'll do all this stuff for you. Every single movement in software engineering and software design and architecture and a lot of stuff is really this super interesting cycle of people's

people, human beings, get this great idea of like, hey, what if we didn't have all of these policies and procedures and we didn't put everything into these boxes and we just said, we're going to make good software and we're going to ship it to people and we're going to own that entire process. And then them doing that and getting good results and then someone coming in later and saying, but what if I could make money off of this process that you have created? And coming up with things like the Deloitte Agile Train,

coming up with AI-based SRE, coming up with observability in a box, right? Like,

Someone will always make money off your complexity. Right. Anytime you introduce abstractions. Yeah. Anytime you create value, someone will try to multiply that, you know, will capture that value for themselves. Charity, I think, has always said this, where tools make borders, right? It's like if this team's using this tool and that team's using that tool, you now have two different teams. Yeah. Which is so weird because as a solutions architect, we would talk to companies, but there were a lot of big companies that you couldn't

the two teams could not talk to each other. They couldn't know that they were both using the same database or different databases. And I'm just like, so like you will pay enterprise support money to get advice when you could just talk to each other within your own teams. Oh yeah, no, absolutely. Consultants make so much money just because they can transcend the borders of the teams. Yeah. It baffles me how like,

I think abstraction and like complexity are all like part of the game and sometimes they can make things better, you know, but like,

It's wild how we are creating our own abstraction and loss of content. Even some ways that we use automation is loss of context. You know what I mean? There's so many ways that we're losing context with abstraction, automation, and just stupid rules that we've put in place. And it's wild because these are humans made this worse for other humans intentionally. I don't understand. So

So I, just by way of background, I used to work for a company called LightStep that a couple years ago got bought by ServiceNow. And it was actually super interesting. I stayed there for a couple of years. And it was super interesting getting to talk to ServiceNow customers. Because if you're familiar, ServiceNow is a big enterprising place and it sells to big enterprising places. Fortune 100, massive multinational orgs.

And talking to them about like observability was super fascinating because they would be talking about like, oh, yeah, we have incident. You know, it's like their incidents would run for like weeks. Yeah. You know, it wasn't like, oh, this is an incident and we fix it in an hour or a day. It's like, no, this is going on. This has been going on for like two months. And I'm like, well,

We're working towards our first nine. We're going to get there. And then you go and it's like, well, talk to me. And it's like, how does it happen? How do you solve these? And the most illuminating thing was talking to someone and they were talking about like, yeah, so this was a telecom company, major telecom. And

I'm like, well, when these things happen, like how are, you know, who's getting alert? Like who owns the problem? Who's getting alerted? And they walk me through. It's like, oh yeah, we have this, they have this very, very comprehensive, you know, incident response system.

process where it's like we have incident commanders and these are all the different steps. And I'm like, it's like, okay, you know, I understand that you have this great process. Cool. But it's still taking you like months to resolve things. Like, how are people getting information? Like, how are people getting data? And it's like, well, yeah, that's part of the problem because, you know, the people that can actually like change the code don't have access to the logs. Like what?

And then it's coming to find out it's like the process involves like teams in different parts of the country, different parts of the globe were literally like because of various security and access controls would go into the one team would push a change. And then the next day, another team would go pull logs and

redact them on their laptop and then send snippets of those logs in a team's message back to the first team. And that's how they're sharing data because the access controls won't let them do it any other way. And it's like, I'm not entirely sure your problem is one of technology technology here. Like just going to be honest, but yeah,

Process is so hard to change, right? This is a heavily regulated company. Changing these processes would require probably tens or hundreds of months in terms of work.

across hundreds of employees across the globe just to make these little changes. And it's like, oh gosh, this is a much bigger problem. This is not something that a tool fixes. Yeah. Well, people are trying to buy DevOps, right? They are. They would love to buy DevOps. How do you then lead them into OpenTelemetry is going to solve a problem for you? So the thing that I actually got out of that entire experience was that

Like what fixes this in a lot of ways are standards. Like technology itself, like you can't buy a tool and expect the tool is going to solve these kind of like very thorny problems because tools come with their own language. Tools come with their own, you know, they make their own silos, right?

But if you can get everyone speaking the same language and you can get everyone kind of speaking the same dialect and you can get everyone kind of aligned and like what's important, what data do we need to figure out those important things? How is that data structured? How is that data meaningful? And can we give the data meaning? Can we have a standard for what gives the data meaning? Then that actually goes a long way towards solving the problem because everyone is kind of

instead of being in their 40 or 50 different silos, now we're all talking about the same thing. And just being able to talk about the same thing and having that shared layer of understanding about what data is available, what format it's in, what it actually means, unlocks a lot of potential solutions for us to get past these kind of human problems. Because if everyone just gets to go and do it on their own, if everyone's creating their own logs, everyone's creating their own metrics, their own traces, their own whatever, then you have...

these understanding gaps that you have to bridge, right? Because my logs are going to be whatever and your logs are going to be whatever. And if we kind of give everyone the ability to go and do it on their own, then my team's going to make a metric that kind of looks good for us. And then your team's going to make a metric that looks good for you. And everyone's going to have their vanity metrics that they talk about.

OpenTelemetry steps into this as a way to unify these various things. It says, hey, here's a standard way to structure observability data. Here's a standard API for the instrumentation of code. Here's a standard set of semantic values that tell you what does the data you're emitting mean. And here's this whole ecosystem of tools, open source, commercial, whatever, that it

plugs into to help you analyze and better understand your systems. And that, I think, is the promise of OpenTelemetry, is that in the future, everything's using OpenTelemetry, and you don't have to think about what logging API I'm using, or do I have a metric or do I not have a metric?

It just exists because OTEL is built in. And if OTEL is built in, then you're getting this stuff for free. And that's the promise. And that is why I work on it. And I'm really excited about it. Now, I mean, there's been standards before, right? Intentional or accidental standards of things like Apache log format, right? Like the Apache log format and Nginx log format, those are assumed standards for a lot of people because everything else just conformed to...

this not great happenstance way of like accessing and grepping for logs. And I was like, I had regex memorized for years on like how to pull out certain things from every single one of those because everything emitted a similar type formats.

But at the end of the day, like that ends up as a, as a, some dashboard that someone's like, Oh, now I'm validating my stuff against the dashboard. But then once another team is like, Oh, we want to use the same data, but I care about different things. They fork the dashboard. And then we had a different conversation still. We're like, Oh, at the end of the day, we're not actually talking about Apache law.

We're talking about the metric that you care about versus the metric I care about and which one's more important. Yeah. I do think there's a couple of important differences in how OTEL kind of approaches this. One is that we have this idea of telemetry being kind of sparse, right? And being highly contextual.

So what I mean by that is we don't want to necessarily emit a bunch of telemetry about the same thing that means that it's completely identical to each other. We want telemetry to layer on top of itself and want to turn it from these three pillars. You know, people might have heard of the idea of like traces, metrics, logs, the three pillars of observability.

The OTEL concept is more like, well, what if all the telemetry was just kind of like this one interlinked braid where it was all self-referential, where my metrics refer to my traces, which refer to my logs, which refer back to my metrics. And I can have this kind of shared context value at a technical level that helps me associate all these different things. And then also going a step further, saying like, OK, there's that core base level of telemetry that all my stuff should emit.

But then I do need ways to process it. I do need ways to transform it for different use cases. Maybe some of it needs to go into different stores. Security team wants something different than an app dev wants, which is something different than a front-end dev wants, which is something different than a DevOps team wants, whatever.

That data needs to go to different places. That data might need to be transformed in certain ways as it goes to those different places. But as long as we're all working off the same base level, and if we're able to separate those concerns, or we'll say like the telemetry is down here, and then these kind of telemetry pipelines are one step above it, then...

it becomes much easier to kind of satisfy everyone's needs while not fundamentally changing the nature of what we're doing like an important part of otel is the idea that we we support lossless transformation

So when you create a metric in OpenTelemetry, you can actually translate that metric into other forms. So you can change from like a delta, you can change like how it's aggregated or how it's displayed using views, using other kind of techniques. And that is not a lossy transformation. You can actually work that backwards depending on how you have your stuff set up. ♪

Okay, friends, I'm here with a new friend of ours over at Timescale, Avthar Suwathan. So, Avthar, help me understand what exactly is Timescale? So, Timescale is a Postgres company. We build tools in the cloud and in the open source ecosystem that allow developers to do more with Postgres. So, using it for things like time series analytics and more recently, AI applications like RAG and search and agents. Okay, if our listeners were trying to get started with...

Postgres, timescale, AI application development. What would you tell them? What's a good roadmap? If you're a developer out there, you're either getting tasked with building an AI application or you're interested and you're seeing all the innovation going on in the space and want to get involved yourself. And the good news is that any developer today can become an AI engineer.

using tools that they already know and love. And so the work that we've been doing at Timescale with the PGAI project is allowing developers to build AI applications with the tools and with the database that they already know, and that being Postgres. What this means is that you can actually level up your career, you can build new interesting projects, you can

add more skills without learning a whole new set of technologies. And the best part is it's all open source, both PGAI and PG Vector Scale are open source. You can go and spin it up on your local machine via Docker, follow one of the tutorials on the Timescale blog, build these cutting edge applications like RAG and such without having to learn 10 different new technologies and just using Postgres in the SQL query language that you will probably already know and are familiar with.

So yeah, that's it. Get started today. It's a PGAI project and just go to any of the timescale GitHub repos, either the PGAI one or the PG vector scale one and follow one of the tutorials to get started with becoming an AI engineer just using Postgres.

Okay, just use Postgres and just use Postgres to get started with AI development, build RAG, search, AI agents, and it's all open source. Go to timescale.com slash AI, play with PGAI, play with PG Vector Scale, all locally on your desktop. It's open source. Once again, timescale.com slash AI. ♪♪♪

When I think of Elasticsearch and it was like, oh, I can make metrics from

All of these logs that I'm gathering and I can then like roll those up into a metric and I'm like going the opposite direction where I'm taking this like really verbose low level thing into like, I want a number now. Yeah. Hotel's not doing that. Right. Right. Hotel isn't doing that part for you. What it's doing is it's giving you structured data that has all of the information you need to do that yourself. So either you as a.

someone creating an observability platform. A good example of this is at Honeycomb, we support OpenTelemetry and we accept all this OpenTelemetry data. Now on the back end,

Honeycomb is designed around this concept of arbitrarily wide events, which is not like OTEL gives you a span, it gives you a metric, it gives you a log message. What we can do though is we can use those hints that are on the data, we can look at the structure as it comes in and we can do interesting tricks to turn metrics into things that are better able to be queried in Honeycomb by looking at that metadata that OpenTelemetry is giving us.

And other people could do this too, right? Like someone could go and say like, just hypothetically, you know, let's say a Grafana or whoever had a metric database and wanted to only allow you to run valid queries on certain types of metrics, like not letting you do a rate for a gauge or whatever, or not letting you, you know,

do certain transformations that wouldn't make sense based on the type of metric it is, then they could actually enforce that by looking at the hints from OTEL about what is this structured data about this metric. Okay, these are the valid query types for that.

Now, not a lot of people do this. In fact, I'm not sure anyone actually does this right now. And that's totally natural. Like, it's going to take a while before the analysis side kind of catches up, I think, to what OTEL is providing. Because OTEL is a very low-level sort of foundational part of this, right? And I understand why vendors and other people aren't necessarily trying to build on top of this fast-moving open-source project. Totally valid. Yeah.

So when I think about like, what does the future look like with OTEL? It actually doesn't look a lot like today. It looks more thinking like, what is the next sort of generation of observability tooling look like? Which is a super interesting question that I don't think we've seen a lot of people start to explore yet. I think observability and metrics are so like underrated and trying to troubleshoot and like really have good insight into what you're building. When you're trying to gather all these different metrics and data and metadata,

Did you run into a lot of issues with trying to keep it like secure, but also to get all that data and put it in one place? It depends. I think there's such a tension between like gathering the data you need to actually understand your system in production and understand like what's happening and give you enough fidelity and like resolution on that data to be able to kind of pinpoint problems and also not collecting too much, right? Because if you get

too much data, then

The way I like to talk about it is like, you know, there's really only kind of three states a system can be in. One is everything is absolutely fine, which never happens, but everything's absolutely perfect and we don't care. Or conversely, everything is broken and we also don't care because everything is broken, right? Now, most of the time, the overall majority of the time, no system is in a everything is perfect or everything is broken state, right? Everyone is, you're always in the middle, especially in like a distributed system or a decentralized system.

Some services will respond more slowly than others. Errors will just happen. These are things that we have to program around. We have to code defensively and have backoffs and retries and pressure relief valves and da-da-da-da-da-da-da.

So what is the right amount of data? It kind of depends on a lot of factors that have less to do with the data and more to do with like, what do you care about and prioritize as an engineering team? Like, what are the things that you are optimizing towards? I think, you know, if you've read Charity and Liz's book on observability engineering, it talks a lot about measuring things from the perspective of the customer.

Who is your user? That's their experience you should care about. And I really like that framing because why also are you building software if not for people? I think people really forget that we get into building the cool thing or building something different or what we want and not thinking about who your actual audience and customer is. Yeah. What was your biggest challenge when building OpenTelemetry, though, as far as a technical challenge or even if it's not technical?

Well, I'm not going to take credit for building OpenTelemetry. It's definitely been a huge effort by, you know, hundreds and thousands of contributors, honestly, right? Like we're the second biggest project in the CNCF behind Kubernetes. Last time I checked, we're like one of the top 25 open source projects on GitHub just in terms of like size.

which is incredibly rewarding and validating just to see how much interest there's been. I think the hardest part of any very large project is just kind of keeping everyone motivated and going in the same direction, especially when you've got something that's... If you think about the scope of OpenTelemetry, we have APIs and SDKs in 13 different languages. We have this kind of tooling ecosystem built in Go. We have...

an overwhelming amount of documentation that needs to be written and updated. We have a really vibrant community, but communities need to be nurtured and you need to give people the ability, you know, ways to grow in that. I think definitely the hardest part has been governance and like trying to keep all that that energy positive and going in the right direction.

Making sure that when people come to work on OTEL, they're working on, like, OTEL is their first team and not, like, whoever they work for. Especially when you're dealing with, you know, in the observability space, there's a ton of companies. Cross-company requirements and demands. Right. And we're doing this for the project, yeah. Right. Just making sure that everyone kind of thinks about OTEL open telemetry first and not their own...

As a systems person, I always saw OTEL as the APM replacement. It's a thing that the devs do in the apps, and it's not for my systems. I know we want logs, but usually I see that being talked about as I want application logs. I don't care about system logs. I know there are OTEL agents or things that emit data from a system level.

Is that still the intention? Like, are we, are people implementing system D unit files with hotel in them built in? Cause there's no, no such thing as like a span on a system D service. Right. So I have different things that I can even do. Where is that overlap going to be? Because when the dev says, Hey, my app is getting this and hotel, I have this metric. It's bad. I still need to correlate that a level below into the, the OS. So absolutely. Open telemetry is intended for like both application and system level. Yeah.

you know, telemetry. We have a tool called the collector, which is kind of like a Swiss army knife that will suck data up from almost anything, including like journal D it will get host metrics, host logs. You know, if you have files and you want to point at files, it'll slurp those up. Var log star, just get it and send it all over. Right. Like it's very, you know, customizable. One thing that we can do is that we can try really hard to like associate, you

In a collector, we can actually associate those particular application-level metrics and logs and traces with system-level data as well. So either directly, like if we have a context value in both places, like a trace ID or something, then you can associate it that way. But we also have ways to kind of associate this concept called a resource concept.

So in OTEL, you have these special attributes that are called resource attributes, and they identify unique sources. So what you can do is you have, you know, imagine I have a Kubernetes cluster or just a fleet of EC2 VMs, and I've got 100 instances of my application running. By looking at those resources and kind of aggregating across resource, I can see like, okay, yes, there was this problem in the app, and

and I see which node it came from, which pod it came from, and then I can also see, okay, what were the system metrics and logs and so on and so forth from that resource? And then if I have tooling for it that can give me a unified view of this, then I can really easily pivot between those two things.

Again, this is something that the tools need to kind of support. You have to have an experience for this, you know, either in a tool you've built yourself or something you've kind of stitched together. We do provide the raw data, but it's up to consumers to kind of like figure out how to take that raw data and to get it into a usable form.

Or do analysis with it. I shouldn't say usable form. JSON is very usable, right? I got JQ. We're good. Yeah, right. It's the data is usable. It's just it's heavy data, right? Like it's dense. I think that's one of the biggest complaints we get about OTEL. And I don't think it's wrong is that it's like super low level. And

It's something that we hear a lot and we're trying to, you know, it's something we think a lot about as a project from like a governance perspective and also like at individual like language levels, like how do we make it easier to use? One thing that we started recently is this new special interest group on developer experience, which is really aimed at like, okay, what are the things that we need to do as a project to make OpenTelemetry easier to use as a developer or as a systems administrator, right? Like where are we missing the mark

Because the goal, I think originally, the goal is that OTEL should be native to your system. It should be native to your framework. If you're using ExpressJS or you're using C Sharp or whatever, it should just kind of be there.

And so we have to write at a low level to integrate at a low level. But a lot of people, when they think about observability, they don't think at that low level. They think more at the like, oh, I just dropped this agent in and it does all this stuff for me. And we have stuff that is kind of like drop-in agent and it does all this stuff for you. But

You know, it's really more of a boot. That's like a bootstrap, right? That's like we had to we had to do this for people to actually use it because nobody's going to go to all the trouble of like writing all this instrumentation themselves. They need the easy button. The hello world for observability. And honestly, like I do think there's a pretty good argument that, you know, if you don't have anything, then something is better than nothing.

And so having an easy button that's just like, yeah, this shows you all of your database calls and your HTTP calls, like that is really useful, especially if you were just if you didn't have that before. I think it's important to have different levels of like easy and then more customizable so people can kind of use their use case and their understanding of your product. Even going back to remembering the first time that like I ran D message on a Linux host and I'm just like, oh, this I wasn't doing anything on the server and it's doing a lot of stuff.

Like, I don't know what it's doing, but it's doing it. Yeah. There's a lot going on. That's, that's everything in software. Like we, it's the good and bad part about,

I reflect sometimes like when I was a child, when I was like 10, you know, when I was like younger than that, I learned basic and I wrote little like a little database in basic to keep track of my baseball cards. And it was fun. Like, and I got older and I wanted to get, you know, I had a Macintosh and I wanted to do like GUI programming. And so I got,

prologue books i got like um c you know c plus plus books and i tried to learn this the the gooey stuff in the event loop and i was just oh boy that did not go well but it was because it was so you know it's like things were hard like even with the libraries and like the the system level stuff the forces it gave you it's still like you had to do quite a bit of work yourself to write a gooey

And now it is a billion times easier. And they work a lot better. You don't even need, you know, HTML, CSS, all this. But, you know, if I wanted to write a graphical program, I don't need to like learn super low level draw, rect, whatever. I can just create a div and say, oh, here's this tailwind class. And boom, I have a box, you know, that makes it easier to use. And that's great. But like all of that creates abstractions that,

We might not necessarily need to completely understand what's going on under the hood, but what is going on under the hood does matter and it can impact performance and it can impact like are people having a good experience. And so there's like a really fine line, I think, with instrumentation and telemetry of how far down do you really need to go to get the data you need to do the analysis to figure out like are people having a good time?

Going full circle there with your GUI Mac apps, your website is awesome. aparker.io is a classic Mac looking WordPress theme, right? Like it's on top of, it's a theme you have that looks like a classic Mac.

And when we were, we had Dave Eddy on a few weeks ago, you could curl his websites. And I love, I love promoting anyone running their own websites. Cause I think that that's still the backbone of the internet is people that have a website and they, they update it occasionally and they put some content there, whatever it is that you want to do, you just do it. Your website is so cool. I'm actually, I'm in the middle of my, my spare time project has been rewriting that.

I started doing this before the entire WordPress thing started happening. But just to go even more full circle, like Justin, we started this conversation. We started, I am here because of a conversation on Blue Sky. The world's, hold on, right? Currently maybe America's number two social network in the app store. I need to check. Yeah. Interesting times in text-based social media.

But one thing that's like super cool to me about Blue Sky is that it's built in this thing called AT Protocol. And AT Protocol, there's two things I really love. One is that it uses domains as kind of like

Identity. Yeah, it's like your domain is like the cornerstone of your identity. And it also is very big on you should have control of your content online. And so I've been rewriting that blog to run off of an AT Proto PDS, personal data server. And it's actually been, it's super interesting because it's one, always fun to try to do

try out new things. But two, like it gives you this really interesting vision of the future where it's like, I could have a single server that kind of, that this is my identity, right? Like this server has all these different things and speaks all these different protocols. Like if I talk to it via HTTP, I get a website. If I talk to it via some other protocol, I get microblog, right? And, and,

You can verify all this stuff because it's all cryptographically signed. And so you can say, like, if someone sends you a link to something, you could prove that, like, yes, this is a blog that I wrote. And if I edit it, you could verify that, you know, it's been edited. I don't know. It's everything's been very static, I think, in terms of the Internet for quite a while.

And I'm starting to believe that people are kind of seeing that the very managed sort of corporate internet that we have all been using quite frequently over the past 20 years isn't really sustainable. That you can't actually run a business off of just selling ads. I mean, you can, but... I mean, but also it's just owning some things on the internet makes it more fun. Yeah, like if you own stuff, like if you have control of stuff, it's good.

Like you, that's the internet. You should have control. So it is. And with that comes responsibility. True. You have to be responsible for what that looks like on a global network of things. So yeah. Yeah. Like I think there's this, we've, we have lost that spirit of like, you know, you think back to the nineties or two thousands and it's like, you go to someone's geocity site and you would see just like amazing things, but

It's why I apologize to anyone that's really big into the idea of decentralized finance, but it's why when you heard about people talking about the metaverse and buying virtual real estate, I'm like, this is so antithetical to the internet. The point of the internet is that you don't run out of space. Yeah, don't make it scarce. Yeah. Except for IP addresses. That was just a bet. Yeah, well, we got V6 now. We're good. Yeah, people try to impose scarcity, I think, on the idea of...

space in the internet is just very um i don't it doesn't sit well with me and i i'm it's why i'm so excited about stuff like at protocol and this idea of a web and an internet that like you invest in this is your space online this is your part of this global network of human beings and

You aren't just renting time from some large company or erratic billionaire, right? I'll be glad to be able to invest in one place and not have to worry about losing everything every time it's hijacked by some rich guy. Exactly.

I mean, the most important thing of that is the place should be the thing you own, right? You're not investing in someone else's place. You have to buy a domain and you invest in your space. And just as Austin said, if you own a website, you are constantly in a state of rewriting it or moving it or doing something else to it because that's why it's fun. Sometimes I think I'll never get it done though if it's like all mine. That's fine. It doesn't matter. It's fun. Yeah.

And I also want to say, like, obviously that's not for everybody. I think it's a great thing. I think it's a totally like I do think that

there should totally be on-ramps for people that aren't interested in that right but you know conversely you don't need a pro not everyone needs a project car not right you can go buy a um corolla and get around some people are just fine with uber all the time right so people are fine with uber or public transit right we need to you know we have to support everyone in their journey but i think that in a lot of ways like that spirit of

This is my little corner of the internet is what needs to come back. I don't see it necessarily as a... I see it as a spiritual thing more than anything else. Take it or leave it, but it really is like you have to have some level of investment. The internet is a global network of human beings and like any network, it is a community. And you get out of a community what you put into it. Don't be a bad neighbor and throw your...

branches up against someone else's fence. I don't know. I feel that. Where can people find you? You can find me around the world at various events. You can find me mostly online on Blue Sky at aparker.io. You can find my blog at aparker.io. What do you know? The same thing. Yeah, the same thing. You can find me on GitHub at Austin L. Parker and

I'm on various slacks and other places. So we'll have some links in the show notes. Thank you so much. I'll send for coming on the show. Thanks for having me. This was super fun. Thank you. It was nice talking to you. Great talking to you both.

Thanks for listening to Ship It with Justin Garrison and Autumn Nash. Subscribe now if you haven't already. Head to shipit.show for all the ways or just search for Ship It wherever you get your podcasts. You'll find us. Thanks once again to our partners at Fly.io, to the mysterious Breakmaster Cylinder for these dope beats, and to Sentry. Use code CHANGELOG when you sign up and save $100 off their team plan.

That's all for now, but come back next week when we continue discussing everything that happens after Git Push.