#30 Apple Vision Pro: A Journey into the Future of Virtual Reality
15:19 Share

hello my friends and welcome back to the future of your ex where we explore everything around new tech design so basically everything that you need to know to keep up to date with the latest tech trends design trends and know what's ahead of us and this episode will be a special one because we are going to talk about the apple keynote

or to say it a little bit more precisely we're going to talk about one of the major things that apple revealed just a couple of days ago which is the apple vision pro mixed reality headset so we are going to dive into the world of virtual and augmented reality taking like a very deep look at apple's first ever spatial computing device the apple vision pro

So friends, grab your coffee, sit back and let's get started. We will get started with a little bit of an intro and then I am going to share with you from my experience or from my point of view the top four things that we as designers need to know or to understand when it comes to the Vision Pro. So my top four insights or thoughts about this amazing headset.

But let's start with a question: What is the Apple Vision Pro? So for everyone who maybe hasn't watched the keynote so far, Apple released their Mixed Reality headset during that keynote. But it's not any normal headset. It's a first-generation augmented reality virtual reality device.

And the most impressing thing for me also during the keynote was that this is this headset is a standalone computer. It's got an M2 chip, its own Wi-Fi connection and a 4K micro OLED display for each eye. Also, let's not forget about the speakers above each ear and a dedicated R1 chip for real time processing from other sensor.

And this is super powerful. It's like having a little supercomputer strapped to your face.

So what can the Apple Vision Pro do? Let's talk a little bit about what this headset really can do for you. One of the coolest features is definitely the eye tracking technology. Imagine this, like you're looking at something and the device automatically knows it, recognizes it. Then you can select what you are looking for just by touching your fingers together. It's like, you know, something out of a science fiction movie almost.

And it doesn't really stop there because the Vision Pro also has the optic ID. So systems that uses biometric patterns in your eyes to lock you in automatically. So no more like family with passwords or pin codes because then there's this pass through mode that also lets you see your surrounding without taking off the headset.

It's like having x-ray vision. It's pretty interesting because you decide the amount of immersion. There's this little wheel where you can say, I still want to see 100% of my surroundings and all the different kind of interface elements are overlaid on top of the real world. Or you are in an environment where you really want to focus or you are in a plane where you just really want to shut out completely and then you turn the wheel and

and the immersion gets stronger. And the Vision Pro also has its own operating system, VisionOS, and web browsing capabilities, so you can even mirror your Mac's display and work with a Bluetooth keyboard and trackpad. And for all who are a bit more on the social side, also Facebook calls with 3D-rendered model of you on the screen work.

Which I think is pretty interesting. I'm seeing a lot of opportunities, especially in the work environment where you really want to focus on certain topics but also come together for Zoom calls, for workshops and these kind of things. So there I am seeing a lot of opportunities.

So the price point is $3499. So I would say it's a little bit more like a luxury business product. It's too expensive for the mass consumer market. But let's see, this is just like the very first generation product. Probably will go cheaper. They also named it pro version. So there might come a cheaper or more affordable version with less functionalities for like the mass consumer market.

Okay, now we are aligned on what the Vision Pro actually is. Let's talk about the top four things that designers need to understand or that are actually interesting for us as designers, right? Because every time there's a new technology coming out and this is definitely like a huge milestone,

For me, it's always interesting to have a look at what does it mean for us as designers, not only for me as a consumer, so I might want to buy it and use it, but also for me as a designer. What does that mean for us? So there are a few things that are really interesting to check out and to understand. So the first thing is the controller-less interaction.

so the apple vision pro doesn't use traditional controllers that you know from the meta quest for example instead it tracks your movement and hands and this means that as a designer you will need to think a little bit like outside the box when it comes to the interaction um

So how does that work? I only have seen any kind of like pre-rendered videos online. So I haven't tried the headset on my own so far. I watched a lot of like videos online from people who tried it out. There are not so many, unfortunately. But it basically works like that. So you don't have any controllers. You only control the whole interface with your eyes. So with your eyesight, with your hands. So you have certain gestures, how you click on something.

And of course with your voice. So voice is another input that you can enter text for example. Which is pretty interesting. For us as designers it makes it I think like pretty interesting to not really think about you know any kind of like controllers or clickable areas but really guide the eye a little bit more.

So what could this involve? Maybe creating larger, more intuitive touch areas or designing gestures that feel very natural and intuitive to the user. It's a whole new way of really thinking about user interaction and it's a very exciting challenge for designers, right? Because this is also what Tim Cook mentioned during the keynote, right?

They are releasing the headset already now, although it will become available early next year. So they're around like seven months or something like that for developers and for designers to already think about apps and think about ways how you can bring your product into the digital OS system.

where you also need to use their interaction methods. You can already think about what are the right gestures to have a really interesting controller-less interaction. So for us, we should definitely explore that and see how that works. Let's move to the second point, which is very closely related to it, which is eye tracking. So the eye tracking is a key interaction method with the Vision Pro. The device knows where the user is looking and can highlight and select

what they're looking at. I watched a couple of videos from people who actually tried the headset and they were all super super impressed how good this all worked. So you basically place the interfaces wherever you want and then you look at certain areas and it for example like highlights it, it also selects certain areas if you look a little bit longer on it and what I have heard is that it works impressively good.

It also tracks your eyes, so it especially renders the areas where you're looking at. Otherwise, it would be too much render capabilities. So it definitely renders the areas where you're looking at. Yeah.

super interesting for sure. So as a designer, really understanding how this technology works is crucial. So you need to consider how to design interfaces that respond to users gaze, right? So really do a lot of testing and research on eye tracking, for example. But also this

need to consider how they design interface that responds to it just like a user case. And this involves like creating visual cues to guide the user's eye or designing interface that react in real time to what the user is looking. And this is also a very different way of prototyping and thinking about interfaces in general. Overall, it's a powerful tool for creating immersive and interactive experiences and also feels

almost like out of a sci-fi movie but very intuitive and yeah like you would interact with the real world as well right let's go to the next point which is the pass-through mode so the pass-through mode allows users to see their real world surroundings while wearing the headset this means

You know, when you think about why some of the other previous products, like the Google Glasses, for example, failed, the reason was that a lot of the users were really shut out from their environment and it looked kind of strange when they were wearing it and they were really in their own space. So what Apple is trying now is to have this pass-through mode where you...

see people who are coming close to you or who are moving or walking around that you can still see them at a glance even if you're in the shutout mode but also that they can see your eyes and the interesting thing about it is that right on front of the glasses they're not actually seeing your eyes but there's a display that are displaying your eyes

which sounds a bit strange, but at least it gives this vision of like seeing eyes and really interacting with someone, although they are wearing the glasses.

So we will definitely need to consider how our designs can overlay and interact with the real world in meaningful ways. This could involve designing augmented reality experiences that enhance the user environment or also creating interfaces that adapt to the user's surroundings. It's a very unique challenge that also requires a deep understanding of both the virtual and the physical design principles. Let's go to the last and final point, which is the VisionOS.

So the Vision Pro runs on a new operating system called Vision OS. And as a designer, we will need to familiarize ourselves with the new system and also its design guidelines. So what I can say and what we have already seen in the keynote is that it looks very familiar to like the design patterns that we are using in web or in mobile. It looks like very, very consistent to me.

But everything has this cool, almost like haptic 3D touch. Of course, there are no haptics involved, but it's like with three dimensionals, but also very almost like transparent blurred background. So I think it's a really interesting thing to see that in your environment. It feels like it really is there. At least this is how it looks like in the promo video.

And we will also really need to understand how to design apps and experiences that feel at home on a VisionOS and that really take full advantage on its unique features and capabilities could evolve learning, of course, new design patterns that we need, but also adapting to new interface conventions or experimenting with new types of interactions and animations. It's definitely a great opportunity to learn and grow as a designer and also to help

kind of like shape the future of virtual reality design. For me, it gets super interesting to see how to really organize content there.

In some of the keynote videos, you see how people are arranging their own working environment, where they're placing certain screens, where they are moving content from one area to the other. Or even one example that they show, which I find pretty impressive, is opening an email with a 3D animation or 3D text.

content piece and then just opening it right away and exploring this 3D animation thing. Could also be like a heart or something for training or education. So very interesting. Also like how the whole interaction really works. Okay.

So for me, let's sum it up. The four really interesting points to have a look at are definitely the controllerless interaction, so how the whole interaction works. Second is the eye tracking. This is part of the controllerless interaction, but something that makes you as a user feel like they're reading your brain. So it feels very, very intuitive. At least this is what I heard. I haven't tried it so far.

then the pass-through mode something that i'm super curious to explore especially that you can like see the real world where there's content on the top and then you decide the level of immersion and then the last point is the vision os the own operating system can't wait to have a look at them and also use that to design a certain kind of

elements, apps to bring all the content that we are having currently on our apps, on our MacBooks, on our PCs to the Vision OS environment. Super excited for that.

So that's a wrap for today. I hope you enjoyed our chat about the Apple Vision Pro. It's definitely an exciting time in the world of mixed reality and I can't wait to see what designers will create on this platform, especially in the next seven months or so. There's a lot going on for sure. So friends, until next time, stay curious, keep exploring and see you in the future.