#65 UX Research Failure of AI Devices - The AI Pin
19:27 Share

Hello friends and welcome back to the future of UX, your favorite podcast about the future, new technologies and innovation. My name is Patricia Reiners. I'm your host and UX and innovation designer from Zurich, Switzerland. In this podcast episode, we are going to dive into UX research and how important it actually is for future and especially AI products. And we will take the AI pin as an example.

Many of you have heard of the AI-Pin. I also did a separate podcast episode about the AI-Pin, a super impressive piece of innovation of technology. And this AI-Pin is the device that was really intended to basically revolutionize how we use tech products. But now after the launch, after it's available to the public, it faces some significantly, yeah, some significant challenges.

And I want to use this device as an example of how important research is and what you can do better. And I'm going to go through five steps, five important main takeaways from this case study. And I would say we are going to dive right in. So first of all, we will talk about what is the AI pin. If you haven't heard about this device, if you haven't listened to the other podcast episode, here's a quick summary of what it is.

The AI pin is a small variable mini computer like a brooch. It's integrated AI capabilities with voice interaction, it also has visual interactions, it can take photos, it can take videos, you have different gestures with which you can control the interactions and you have a projection of an LED visual like LED lights basically on your hand.

And when they introduced this device, they had some amazing scenarios prepared. For example, you are on a market, you want to buy something. You have this AI pin here on your shoulder. And then it sees basically all the fruits and vegetables that are out there. You click on it and then you ask, what kind of vegetable is it? And then it says, oh, it's a broccoli or something like this.

So it can basically see what you see and give you feedback on that. And with the camera and the microphone, it should provide an interactive communication. So the thoughts behind it is that it basically sees your surrounding, it guides you through everything and you have a constant way to communicate. So people were super impressed when they saw that. I also was super impressed. I thought, okay, that's awesome.

But the truth is when you compare that to the reality and to how things really are, yeah, there's definitely a gap. So they face some challenges. And now this device is out there and it's not that amazing as it has been presented in the promo video or in the tech videos that the company Humane published before.

For example, this device has a super slow response rate, especially when the wi-fi is not ideal. So when you click on it and ask a question, it takes sometimes up to 10-15 seconds until you get a response back. And with your phone it would be much much faster. And we all are known, also users know, that it usually takes around less than a second until you get an answer.

Then there are a lot of frequent misunderstanding and errors in the AI responses. So they're just not correct, they're not right. There are a lot of funny examples on the internet. And there are also some hardware issues with overheating and uncomfortable weight. So when you put it there on your shoulder, basically on your shoulder, like a little bit more in front, it's very heavy. So it pushes also your clothes a little bit down.

and it of course depends on cloud connectivity which leads to inefficiency in data processing. The feedback from users hasn't been so great. A lot of frustrations with the device operational delays and inaccuracies, issues with the physical design, the discomfort, especially with the varying internet speed and external conditions can have a huge effect on the performance of this device.

So the user expectations based on the promotional material on the videos versus the actual utility and efficiency on the device, yeah, doesn't really add up. So this was a little intro about the AI pin. Now let's dive into the second point, which is the user experience challenges. We all know how important user experience is. We know how important it is to design around the user and not just add features on features and features.

One thing that every user expects is to have a seamless interaction, intuitive user interaction with modern tech devices that can be the AI pin, this can be your phone, it can be basically any device. Just because it's new and innovative, people don't have suddenly more patience and are happy with more delay time. So when it comes to a good user experience, and especially you think about new technologies

ways to interact with, for example, with gestures, which has been a quite innovative way that you use the palm of your hand

as an interface, right? If you just think about that. Because you don't really have an interface, so what could be something? Of course it could be the palm of your hand, right? Where you see a little projection only with light, so this is basically black and white or green and bright. And you see there you can switch between different songs for example, you can see the way, where to go and you can also use your fingers basically to interact with it.

But it's also not super easy to see at all time and all conditions. Especially with different lighting situations it's not that easy to read. Or depends on the interface that's projected on your hand. Not every hand has the same size, some hands are a little bit smaller. So it's very easy just to read the typography at certain parts. And of course you don't have any privacy so basically everyone can see what's projected on your hand.

And I think this is such an amazing example of how important it is to have real world testing conditions.

There's a huge difference between testing these kind of products in a lab with the perfect Wi-Fi, with perfect lighting conditions, with a super calm surrounding, than testing it in the real world scenarios. Also with not super tech-savvy users, with like normal people who are using that for day-to-day work. They would have noticed that the Wi-Fi is very slow, that it takes...

so long to get responses that some things are not working. So they would have seen that if they would have tested in real-world testing conditions. And I think this is also a super important learning when you think about AI. It's so easy to think about like, okay, let's make something super crazy and super innovative and super new.

But the truth is, everything is about the user. If the user doesn't like it, although it's super innovative, it won't be a success. And this is what we're seeing with the AI pin. It's an amazing piece of technology. I think it's still super, super impressive. But seeing that with real users helps us understand that this won't be a success, this product.

And for me, kind of unbelievable that they obviously haven't tested that or they tested it and they thought it's still okay. I would have rather pushed this whole thing back until summer, for example, than putting this out there. Let's move to point number three and I think this is also super important and I think something that everyone can relate to. Nowadays, there's a lot of technological overload, a lot of feature overload.

What might be much much better is really simplicity. I don't think that more is always better. So maybe you don't need the features of the AI pin such as like all the gesture control and the AI integration standalone connectivity. It can be all a little bit too much. So what might be helpful is to first start on one way of interacting and then introducing more ways.

But of course I understand why they did that. They wanted to have also like the huge bus. I mean there has been such a big conversation around that topic. Everyone talked about that. I also talked about it because I think the technology was so impressive and what they presented. And then it's especially frustrating if you see that the reality doesn't really add up to what they promised. So people really love simple devices.

But what really doesn't work is if they have a device and they think this would maybe replace the smartphone for some times and for some scenarios, but it actually doesn't. So what makes a little bit more sense is to think about certain use cases, for example. Could be sport, for example, running or sports.

cooking or something like where you have like certain activities where you can really use the phone and then create experiences around that basically or create the features for an AI pin for example around these kind of experiences or these use cases for example. When you think back about the first story that I told of the market you know where someone goes through the market and then he has this this pin there and asks like what kind of vegetable is it I mean everyone knows how broccoli looks like but maybe it's something more exotic

And you could also use your phone, right? You have your hands free, there wouldn't be a problem to use your phone. But maybe you have other activities where you can't use your hands. So that would be something super interesting to think about, right? To think about these. They're not really edge cases, but I think that really makes sense. And this is something that actually adds value.

So this is point number three. It's think less about the technological overload and all the features. And this is a typical problem that I'm seeing in so many tech products. Adding another feature, adding another feature, adding another feature and then leaving the user completely alone and making them deal with the technological overload with the feature overload instead of keeping it very simple and easy.

Okay, and then point number four, and I think this is also a super interesting one, is this money. So the AI pin costs around $700 plus a monthly subscription fee of $24, which is a lot actually. This is almost as much as a smartphone, not an iPhone, but as a smartphone. So when you compare the functionalities with a smartphone, it's definitely not as good as a smartphone.

And for 700 dollars it's actually not that great. Although the technological aspects are great but the usage not so much. So I'm not sure what do you think? Does it really justify the high costs in the eyes of the consumer? I don't think so. And especially having this ongoing subscription fee something that really adds up over time.

And if you compare that to... I think comparing that to the smartphone is probably not the best thing. But if you compare that to the Rapid R1, I know this is not a brochure, not something that you pin on your clothes. This is also something that's AI first and AI first device and I think super innovative and interesting. And this costs $199, so much less. Let's see, we haven't tested this device. I'm super excited to do that.

But in the meantime, I think there is definitely a huge gap. So when we think about innovation, it's super important for us as designers to think about or aligning the technological innovation with the economic reality and user needs. The economic reality is that a lot of users probably won't have the money to spend $700 on this device

as if this is an additional device to their smartphone. This is not a replacement, this is an addition to something like a little tech gadget. So we really need to think a little bit critically about the cost versus the benefits of these new technologies before really adopting them. Let's move to point number five and this is the future of personal assistants. I mean we are all having personal assistants now using large language models like JGPT, Google Gemini for example.

And we are also going to have devices that are more seamlessly integrated into our daily life like the Rabbit R1. I think this device has a lot of potential, super excited about it. And generally also the Humane AI pin. The problem is that

It's an amazing piece of technology, but it's facing tremendously problems, especially when it comes to the user interface issues, the AI responses and the physical design problems and the user needs. It doesn't really line up in the end. And this is a big problem. This is a first generation device. I'm not sure if there will be a second one, but...

I think the whole experience around that really shows that it's about the user and the user and their understanding first and their needs first.

and the importance of user-centric design and testing. And especially when designers think like "oh there will be a future and we don't need research, we don't need designers, we can automate everything" - we definitely don't. And this is such a good example of how important user research actually is and how much it was lacking in this project. I'm sure they have done research, I'm sure they have done testing but not proper research or testing.

Otherwise they would have seen these things and would have never brought them out there. So there is also the goal for us to balancing innovation with practicality. I mean, all the problems will be coming better at a certain point.

But topics like privacy and security and the development of these personal assistants are super important that not everyone can see our interfaces, can interact with them. When you have them on the palm of your hand, for example, imagine you're, I don't know, in the bus or somewhere where there are a lot of people around you, they can basically see everything. With the smartphone, I feel like you can turn it a little bit to different corners that not everyone sees on your screen. But if you have it in the palm of your hand, it's a bit tricky, right?

Thinking about the future, this also could be something that could be projected on your AI glasses or contact lenses at some point. So there will definitely will be an amazing future of these AI assistants, of these little helpers. I'm not sure how they're integrated. The smartphone is a like transitional technology so this is definitely going to change but

the AI PIN just thinking about this device won't replace your smartphone anytime soon so we don't need even need to think about it but personal assistants will be the future and now we just really need to figure out how to integrate them what kind of devices are the right devices the moment the phone is still the best solution for that but and i definitely want to highlight that user research is essential right

Okay, so I'm going to summarize the key points real quick. First of all, the clash between the concept and the reality. This was like the big learnings from the AI pin and the user research case study. The reality and the concept really, they clash. They don't really match.

They promise a lot of things and the reality looks very very different. Then, point number two are the user experience challenges. Some things that the user are facing and that shows that they haven't done proper research or that there has been a problem with user research. Probably because they have done this in labs or in close surroundings with perfect Wi-Fi with perfect lighting but not in real time scenarios, in real life scenarios.

And point number three is the technological overload versus simplicity. They're trying a lot of different things at once, but what sometimes might be a little bit more helpful is to focus more on certain niche target groups and make it a little bit more simpler.

Of course, they probably had their reasons to do that because of probably investors or money, but still generally thinking that would have been a better decision, I think. And number four are the economic and practical viability, thinking especially about the monetary aspect. Very expensive device. It is an add-on to your smartphone, so there's a lot of money that needs to be paid to use this device. This also makes the target group or...

people who are buying these things much smaller. And number five: the future of personal assistants. I see an amazing future of personal assistants currently there in your phones with large language models or on your computer. They will be integrated much more seamless. How this looks like we will see.

So this has been a little intro or a little case study around user research and the AI pin. I think it's always very fascinating, especially from a UX, from a design perspective, to have a look at these devices and technology and really see how they are evolving and also seeing a little bit like the pitfalls of these new devices. And that is not that easy to come up with good innovation, but also that you

the user is in the center of everything and we need to design around the user. And this sounds so abstract, but I think this is the perfect example of

How important user research is. Okay, amazing. If you liked that podcast episode, if this was helpful to you, I would be so happy if you could give this podcast a five-star review. It would help me, it would help my team to, yeah, do the podcast and promote it, give it a little bit more awareness. And thank you so much for your support. I really appreciate that. And now, wish you all a wonderful day and hear you all next week.