300: The Aftermath
29:11 Share

Welcome to Under the Radar, a show about independent iOS app development. I'm Marco Arment. And I'm David Smith. Under the Radar is usually not longer than 30 minutes, so let's get started.

So the sort of summer series that we've been doing here on Under the Radar has been all around the Overcast redesign, the preparation for that, the technical side of that. We did an episode that was sort of the getting ready to actually launch it. We did an episode right after you launched it. And I think it's probably a good point as we wind down the summer, we're starting to get into...

where there's going to be new things, you know, the first Apple event of the fall has been announced. Things are heading in that direction, but I think it's good to bookend this summer with a follow-up. You know, we titled the early posts, we had like the storm before the storm, then we had the storm, and I think now it's kind of probably a good time to have the aftermath after the storm. So this is the kind of checking in on how the Overcast, you know, big redesign launch went. And it's, you know, certainly from the outside, it's been an interesting thing to observe, but

Because I use Overcast every day, many times a day. And I, you know, business on the beta, I've used it. And now it's just what Overcast is. And it's I'm very used to it. It works, you know, very well for me. And I use it all the time. But, you know, from my perspective, it's great. In some ways, I've like moved on to this is just the new normal and everything's great.

But I get the impression from talking to you and hearing you talk about Overcast and ATP that the launch has not fully settled to that place for all of your users, that there are many people who are still having difficulty adjusting to the new reality of the new redesign and some of the changes you do there. And there's definitely been some things that you've had to go back and revisit and change and make use of the thing we've talked about many times where now because it's all in Swift and SwiftUI,

you're able to work and develop much more quickly. And unfortunately, you've had to put that to the test right away because there's one of these things that you had to work on as quickly as possible and change. But I think before we get into the technical side, I think just initially it's probably good to just start off saying it's like, how are you doing? How are you feeling about the update? How are you know, how is the last month or so, you know, been treating you? It has been a roller coaster. It has really, really been a lot of ups and downs. And

You know, when I was early beta testing this rewrite, the feedback I got from my testers, including like, you know, when I did the very early test, it was only like, you know, 30 or 40, like, you know, friends, basically. Even that feedback was mixed. I knew right from the start, this is going to be a bit of a bumpy ride. I was changing some things, I had removed some things, and I knew, like, this is a risk. And

And I knew I had to do it for all the reasons I've said before. I knew I had to do this rewrite. It kind of had to happen soon because just more and more time was going by where I just wasn't able to really update my app and serve my customers. And I was getting emails every single day from people saying, like, this app is so slow. The old app. It's so slow. It keeps locking up on me. I have a large collection. It doesn't work. All this user feedback and negative reviews every single day.

you know, citing problems that I had fixed in the rewrite. And so I did feel some, you know, some time pressure to say, you know what, I got to get this out this summer. Like I can't just let it wait another six months. Like I have to get it out this summer. And so I, you know, I cut scope and made it happen and I got it out there.

And I knew right from the start this was going to be controversial in certain ways. The streaming removal, certain design changes. I had no idea what I was in for. This is by far the longest I've gone without a redesign before this. It had been a few years. And this was the biggest redesign I've ever done in the app. And, you know, the app being 10 years old, and I think that this is the part that I think I underestimated. To some degree, it doesn't belong to me anymore.

Sure.

In some ways, it kind of felt like George Lucas messing with Star Wars. It kind of felt like, oh, no, I am messing with something. These people hold dear and use all the time, and they've had it for so long, it has almost become theirs. And who am I to mess with that? And I think I underestimated...

responsibility in that area and I overestimated how opinionated I was going to be able to be with the update. Like, if I was making a brand new app from scratch, I would be able to be a lot more opinionated because people could choose to use it or not. But that's not how App Store updates work. And this wasn't making a new update from scratch. This was updating an existing app that had an existing user base that would be forced to come along with whatever I'm doing. So,

so I was making decisions about the app that I thought were like bold and forward-looking and opinionated like to a good degree but I think I overplayed that angle and I think I went a little too opinionated a little bit too bold I was making decisions more like it was a brand new app not an update to an existing app that was a decade old and had a bunch of users already who you know we're using it every day so I

I think I overplayed that. I think I overreached with my changes. And so much of the summer has been, you know, some bug fixes here and there, of course, because, you know, anytime you ship a giant rewrite, you're going to have bug fixes that you got to do. And I still have some of those I'm still working on. But a lot of the changes have been basically walking back some of my opinionation, whatever that word would be. Sure.

you know walking walking some of that back to say like look i thought i could get away with removing streaming turns out i can't like that that was a big one that was a very big one um i can't get away with that i thought i could get away with you know certain minor you know option or behavioral removals like one tap play with you know this mode i've had since the beginning of time um i thought i could get away with that i figured how many people possibly use that uh turns out a lot the thing is with the app store like the the star rating system matters a lot

In the sense that not everyone, in fact, I would say almost no one seems to actually read written reviews. And the way the App Store displays written reviews, where it basically has like one or two big blocks that show like the top one or two reviews, and then the rest are basically buried. I think written reviews are not very useful to most people on either side. But star ratings are.

Star ratings are displayed everywhere the app is displayed, everywhere in the app store and in every location. Next to it is a star rating average and count. And so for that number to go meaningfully down will really hurt your business. It matters a lot. So I'm very sensitive to that. And I've been watching this graph on app figures of the average of new ratings. And that has been...

to say the least, that I was able to see, like, all right, before the rewrite, my average of new ratings was, like, in the high fours, you know, 4.5, 4.4, 4.6, you know. And then after the rewrite, it was down to, like, 2.5, 3. Like, you know, it was substantially down. And I'm fortunate that I've never actually reset my ratings in App Store Connect. Like, you know, you have that option when you upload that you can have the new version start fresh with no ratings. The world's scariest checkbox. Yeah.

Yeah, the reset summary rating feature. I've never used that. So I have 10 years worth of ratings built up. And so I was lucky that like,

mathematically speaking, the average was still very high because I had 10 years of high ratings even though I had two months of pretty bad ratings. So the math was still working out, but that was a clear sign to me, I have to do something. I gave the metaphor many times of it's like a building on fire. It's a very small fire on a very big building, but I need to put it out. That's going to be a problem if I don't. So

I really have been kind of lost for a lot of the summer trying to figure out like, what do I do to please people?

So re-adding streaming, that was a big one. That was probably the biggest thing that I've done that turned that around a little bit. A couple other minor things here and there. And what I'm working on now, which should be out soon, is a swipable UI for the now playing screen to toggle between the main controls and the info. Similar to what we had before with the cards, but better with using the whole screen.

That should be out soon. I suspect that will also kind of win a lot of ratings back from the depths of twos and threes or ones. So that'll help a lot too. But I think like overall, I have actually felt pretty bad about this for much of the summer. It has been a pretty substantial stress point for me.

I have often felt negatively. I've often felt like, you know, screw these people, you know, which is not a productive way to think. And then it's just, you know, you gotta, you know, feel that, tell yourself that, and then move on whatever way you can, because that, that leads nowhere good. But, you know, then you gotta, you know, you gotta get your head back in the game and, and just fix the problem. And so that's, that's what I've been trying to do. You know, I could tell that like the rewrite was a very bumpy launch. There,

there's i i've learned about like there's a whole reddit about people who like really hate me um which i didn't even know was there until a couple of weeks ago and i found it and i took a look i'm like whoa tip for indie developers don't go to red if someone makes a reddit about your app don't ever look at it um it's it's not good for any of your mental health um

What I've been trying to do is try to look at feedback channels that are not full of personal attacks on me as a person. And so that would be not looking at Reddit, but looking at things like my rating average. When you look at metrics, look at engagement, look at things like how many sales are you getting? What's your revenue? If you have an in-app purchase, how are those rates looking? For the handful of places that are full of people writing things about me, your written app store reviews, places on the internet,

The message there is substantially more negative than what your metrics will typically show. And thank God that is true for this too. Because if all of my input was as bad as some of those human written ones, I would quit the business. It's awful. When you're getting feedback about your apps, look at the ones that are more...

uh, objective and more data driven. So are people using the app? Has, has usage gone down? Has, you know, have purchases gone down? And fortunately in my case, all of those other metrics are, are either level or increasing. So every other metric I have to say like, is my app healthy? Did this launch go well? Are people still using it? Are people mad? Are people on fire? Am I terrible? Is my app terrible? Every other metric says no.

And thank God for that. Because again, I would have a hard time continuing if that wasn't the case. But every other metric is positive. So you just have to look at like,

How do you manage these inputs? How do you manage these feedback channels? And obviously, some of them, people are making good points. Some of them, people are complaining about real bugs or real shortcomings. And so you take that as input as you can. But certainly, there's a balance to be struck. And I've had a lot of trouble striking that balance all summer. But I think I'm coming out of it now...

in a better place both psychologically and with the app in a better place. And I'm not out of the woods yet by any means. I still have some bugs to fix that are pretty important that are affecting a lot of people. I still have the interface changes that people are asking for, that enough people are asking for that I think I need to do it. So there is a lot that I still have to do, but it's...

It's getting a lot better now than it was, say, a few weeks ago. Sure. And I think certainly the trend there, I mean, the most important trend is how you're feeling about the happen out of those things. Ultimately, certainly from my perspective as your friend, that's the thing. It's hard to hear how rough...

the last, you know, months or so since, since it launched have been in that way. Cause that's, and it's, I mean, I know that I say that because it's, I know very specifically the way that that can feel that I think one of the wonderful joys of being an indie developer is feeling a deep sense of ownership and

of the work that you do. And there is less of a gap between the work and you as a person because you're putting yourself into your work. And many of, you know, all of the success is yours and all of the failure is yours is just the reality that it's really cool that you can feel

You built this thing from file new project years ago, a decade ago, to what it is now is the result of your efforts and your labor. It wasn't a situation where you were working – if you worked at a larger company that had 20 developers and you spent your summer working on –

the audio engine. And, you know, it's nice when someone says the audio engine is good and it's nice when, you know, things go well in your department. But overall, if the app is doing well or not, is this kind of diffuse thing. Whereas for you, I imagine this is the feeling I often have and I've, you know, sort of had to navigate is it's

I feel tremendously connected to the work because the work was created almost exclusively by me. And so it's really hard when things, you know, when people say unkind things about you or about your work, it feels very difficult not to take it personally.

Oh, yeah. Yeah. And so it's not something that you can easily just like turn on and off in that way. Or once you see it to just ignore it or forget it because you're so connected to it. And so I'm, you know, it's one of those I'm glad that it sounds that the as much as that trough in the middle is awful.

that perhaps we can be looking forward to be climbing out the other side at this point, that we can sort of move forward with the fun parts of this and the exciting parts of this. And maybe you've lost some customers over this process, but I think there is something that I find somewhat reassuring when I look at the app now is the app itself is, I think, objectively better than it was before the update.

And the reality of that, in terms of it is faster, is more responsive. It behaves in ways that modern apps do. It's being updated much more frequently. There's all these things that make it better. And those are the things that will benefit every time you get a new customer. And as much as it's useful and wonderful to have an established user base, in some ways, the success and the long-term viability of our apps...

is very connected to the newest user that we have. That if we get a new user, do they stick around? Are they there in a way that is going to make use of the app and be retained?

Because if they aren't, then the app is just sort of on this inevitable slow decline that won't go anywhere. And I think what's great about this is you're now in a place that the marginal new user every time that they download the app should be in a much better place. It should be, oh, wow, this app is better, it's fast, it's choppy. And they have no baggage around the things that have been lost or changed coming from the old version. And so in some ways, in that sense, it's great. But the process of getting there is just...

is rough and it's complicated. And there's no easy answer to with so many of these things, because I mean, something that I always struggle with, and I think I've seen bits and pieces of this in this update is it's so easy to be like, well, why didn't you just write like that? And then it's like, well, why did you get rid of streaming? Or why didn't you just make it so that it does sort of where you ended up now? You don't have true streaming, but you just download when a file is

We hit play before it's downloaded. It starts downloading it and then starts playing it with whatever it has while it has it kind of a thing.

And it's like, it's not that you got rid of that for no reason. There are other reasons and trade-offs. And in all of these, one of the fundamental realities of development is you have to decide if you're going to do this or that. It's a bad path to only say yes to everything, that you're just like, well, I'll do both. I'll have this and this and this and this and this and this. And then you end up in a place that the app has no personality. It has no value.

opinion in it. And I think, so in this case, like streaming,

The way you ended up, I think, works really well. I think it checks the box enough that if you're solving the actual problem that people were having, which is usually, oh, wow, there's this new episode of a show that I wanted to listen to. Oh, I'm excited. Let me hit it. And it starts to play. The thing that was probably, at least my guess is, and this is from my own experience, the thing that was really the frustration people were having is they were excited and then they were disappointed. Right?

And like that distance is the thing that you need to address, not the ability to arbitrarily, you know, zoom back and forth between things in ways like the broad version of what streaming would mean. But, you know, it's like you got rid of it for a good reason. And I think as a user, I'm glad it's back because there are situations where it's I very much appreciate being able to start listening to something right away.

But it's tough because you took it away for a reason and in some ways for reasons that are outside of your control, that the dynamic ad insertion and all these other things that are complicated in making a podcast player mean that you sort of had to do this in order to keep other features going. And so it's tough. So you ultimately just had to make the decision. And in this case, unfortunately, you had to sort of

But unfortunately, that's the world we live in.

Well, and again, I kind of get why people act that way because they feel powerless. Because here's me, this faraway developer. A lot of them don't even – well, the ones on Reddit, I guess, know me. But the ones in written reviews often don't know that this is just one person or whatever. They feel powerless. They feel like this app that I've been using has all of a sudden changed. I had no say in the matter, and it works for me in way XYZ.

And so, of course, they're going to respond strongly. I totally get why that happens, and it's kind of unavoidable to some degree. But with streaming in particular, that was obviously a big one. And the reasons I wanted to get rid of it, I just now...

it's it's very technically expensive to support streaming i don't mean computational power wise or money wise i mean complexity wise and like what apis i have to use what apis i can't use because i have to support streaming like there's there's a lot of complexity there on the on the lower level as well that's why i was thinking like well dai makes it kind of kind of break in these different ways and it's super difficult to support technically and it's

it restricts what kind of APIs that I can use in the audio engine as a result. Wouldn't it be great if I get rid of it? So like, you know, there was, there was a lot of thought that went into that. But again, you know, I, I made the wrong call. Like, yes, it is very expensive to, to, to support streaming, but it turns out it was more expensive not to. And so that's, that's the process I had to go through, you know? And,

And again, I understand... I don't blame individuals for saying mean stuff or anything. Because again, I understand where people are coming from. I understand why they're mad. When we criticize Apple for something, that's not obviously an individual, but...

We criticize Apple oftentimes not because we hate them, but because we like them and we like their products and we want to keep using their products or their products serve our needs the best overall. But there's some flaw, X, Y, Z, that is making it hard for us or that we know they could do better on. And why are they doing badly on this one detail? So we complain about it in public. And sometimes we say unkind things because we're frustrated because these are things that we like and use.

and depend on, and they change in negative ways. So I totally get that dynamic. And so as a developer, you can't really fight that dynamic. It's a good problem to have. One of the general categories of shortcomings I've had with this update is there will be some little behavioral detail in the old app that is exposed in a use case that I never do or that none of my testers did in my beta test group

But that some percentage of my user base does that, and that little detail is now different or missing or broken in the rewrite, and it ruins their workflow in some way. When you have an app for 10 years, especially, I mean, podcast apps are...

really deceptively complicated. The amount of little workflow details and small implementation details that can be different between two podcast apps is just massive. And so even though if this is a minor thing that I didn't even know about or didn't think mattered...

If some menu item somewhere gets moved or changed or deleted, suppose that really angers or really breaks the app for some tiny percentage of my app user base. Well, that tiny percentage of the user base could be 1,000 people. And if many of those 1,000 people go leave one-star reviews, that makes a dent. That makes a big dent. So even though it's only a small percentage of the user base...

If you anger them enough, they can cause big problems for you. And again, this is one of those things that like I underestimated the impact of this because I was still thinking of this as, you know, last time I was making major opinionated decisions about this app, it had way fewer users and it was a long time ago.

So I really underestimated that. And so that's, again, leading into all of this is like things I did not fully appreciate going into it, but I think I'm in a better place now.

We are brought to you this episode by Sentry. Sentry are the folks who help you avoid your mobile app being the one that gets deleted next. Boy, this hits home. From getting deep context into crashes to seeing full insights on performance issues, Sentry is here to make sure your code doesn't suck. Fix what's broken faster and keep your app running smooth with application monitoring built for mobile developers. This is Sentry.

so important as we've been discussing for this whole episode for both your apps and you see in other people's apps too. You need to know things like, is your app breaking? Is scrolling stalling? Do you have weird latency? Are people rage clicking in your app? It's super important. So you got to keep your mobile users happy. Otherwise, they will go ruin your star ratings. So the great news is

Sentry is totally free to use. So there's no reason not to check it out. Go check it out. It's really impressive. You got to see Sentry. So even better, as a listener of our show, you can get six months of the team plan for free. So click the link in the show notes and use the code radar. That will get you six months of the team plan completely free at sentry.io slash four slash mobile.

or click the link in our show notes. Then use the code RADAR so that you can come from our show. Our thanks to Sentry for their support of this show and Relay. So something you said there that I think is really helpful to sort of wrap up why this summary is so difficult, I think is you were saying some of the things you could only learn

after the fact, right? You learn the value of these features to users when you take away the feature from them.

And I think it is very difficult to understand those effects of a change because when you're making the decision on the first side, you have all of the reasons to make the change. And you're very aware of them and you see them and you can quantify them and actually wrap your head around them.

But there's all this information in sort of this equation that is just completely impossible to understand until you really get it out there to the broad audience that, oh, you could have a beta group. It's like, sure, but is that beta group representative? No. Spoiler. No, it's not. Do most of them even install it? No. Yeah.

And even if you even if you had a representative group, it's like, well, OK, well, now you have two problems. Now you have thousands of people with different things. And it just gets like the reality of any situation is that there are some things that you can only learn by actually doing it.

And you can try and guess. And I guess the experience that you and I have have of more than a decade of being developers is hopefully our intuition is good for what kind of things are, you know, generally going to go down well and which ones aren't. But the reality is, you know, even with more than a decade of experience and the benefit of, you know, showing the app to other people who have even more experience, there are things you will only learn once you have it out in the world. And as a result, you know,

you can't make a fully informed decision. And so there's always going to be situations like this. And in some ways, it's like I find some comfort in that in my situations where I've run into this in so far as it isn't that I made the wrong choice in so far as, you know, I was I had all of the information and I sat there and I thought through it, you know, in the way that many decisions in life we tried to do. Instead, it was much more of a situation of we made a guess as to what the implication would be.

That guess was wrong because we didn't have all the information. And so now we can hopefully adjust, change, and improve going forward. But it makes it slightly more palatable to me insofar as it isn't – it's a mistake insofar as the impact was larger than we wanted it to be. But it isn't in the sense of, you know, we were being foolish by doing this. Because I think many of the choices that you have made here were very wise and great and it made the app better.

And, you know, in that sense, hopefully, I don't know if that gives you any comfort, but it certainly gives me comfort when I think through these because it's just, that's the reality that this information only exists once you hit release in App Store Connect. And until then, it is all speculation. Totally. I mean, like, you know, I've seen a lot of, a lot of the frustration from people has said things like, why didn't I do a bigger beta test or why didn't I release it later? And the answer really is that neither of those would have changed anything. Like beta testing is, is,

is useful up to... It's almost like when you're taking polls of people, you get most of the gist of the results with a surprisingly small data set, or a surprisingly small sample set, rather. And then you just keep adding more people to it, and the percentages don't really shift that much. When you're beta testing, you get...

a lot of good feedback from a fairly small group, the more people you add, you don't really uncover that many more issues. Because, again, the beta response rate is not great. You can have a beta test. I have a beta test officially in my Test Flight group. I think I have 5,000 people in it. But the average install rate is, I think...

maybe like a hundred people who actually install it. Like people just click the links, they join the test flights, they bail out. Like it's not a reliable process beyond a certain size. You know, you have like your core audience, your core testers who will reliably test things. It's a pretty small number generally. And then everyone else is kind of just not doing anything. So you're right. You just got to get stuff out there and just try it and be able to make changes. And that's why, like, as you mentioned earlier, this has been massively beneficial for me

To have the new code base, to have this... It's much more nimble. I can be much more dynamic with it. You're going to see when I do the swiping update, like, that's a major change that I was able to do in, like, two weeks just because it's all better code, better foundation, easier to manage, easier to maintain. So...

It's been a very bumpy ride, but I'm coming out the other side now, and I'm finally hitting my stride. And for all of you out there who have inconvenienced during the summer, I'm sorry. Thank you for your patience, and it's getting better. So thanks. And the great thing is you're not out of the woods, but you can see the light between the trees. And I think that is a great place to be, and hopefully going into the fall, that's...

you know, just gets better and better and the light gets brighter and brighter. Thank you. Yeah, I'm glad to be there finally. So thanks everybody for listening and we'll talk to you in two weeks. Bye.