2024 BAVP: Before Apple Vision Pro, there were virtual headsets that made many users suffer from nausea and disorientation. Having worn them, you would have had a tough time finding a date.
2024 AAVP: After Apple Vision, there is the promise that VR headsets will be the future. It’s futuristic and geeky cool.
The two words that will define the next generation of technology: Spatial Computing. The Apple Vision Pro has launched in the US and it’s only a matter of time before it begins rolling out to other markets. There are two reasons why the headset didn’t launch in more markets simultaneously — the production process is new (and expensive) and users need some degree of handholding.
Let’s get a few things out of the way, namely size and form factor. It looks like a pair of expensive ski goggles but with every generation, the headset will become slimmer and lighter, like any other gadget. The early cellphone weighed more than two pounds and looked as sexy as a doorknob. Second, pricing. Given the amount of technology that has been bunged into the headset, it’s tough to complain about the pricing. Plus, it’s a new product category that will have cheaper versions in the coming years.
What is the “thing”?
Apple calls it a “spatial computer” but you can also call it a VR headset. The idea is to blend digital content with the physical world while allowing users to stay present and connected to others. A disclaimer: We haven’t tried on the headset, because it’s at the moment only available in the US. So, we are going to focus on the whats and whys.
Apple CEO Tim Cook reacts with customers as Apple's Vision Pro headset is displayed at the Apple Fifth Avenue store in Manhattan in New York City on February 2
Some have seen a VR headset before and perhaps even used one. It’s got displays, lenses and speakers but Apple’s treatment of “spatial computer” is unique and there are no gadgets that match it.
When a company comes up with a new product, there are goals for it, like the features that would come in a year’s time, two years, three… you get the idea. The same happens with the iPhone: The world is expecting the iPhone 16 later this year but Apple must be working on, say, iPhone 18 or 19 or 20 and how those work with its other products. Both five-year and 10-year roadmaps are necessary. Since Apple is calling it Vision Pro, there is always the implication of having the device without the ‘pro’ moniker. That itself opens up a lot of avenues.
Immersive fidelity
Over the years, I have had the opportunity to use a bunch of VR headsets but all of them had too many downsides, particularly in terms of immersiveness, discomfort and limited appeal.
The Apple Vision Pro has the sharpest micro OLED display you can buy. The system packs 23 million pixels across two displays, each the size of a postage stamp, with wide colour and high dynamic range. The size of individual pixels on these displays is seven and a half microns, which means you could fit 64 of them in the size of a single iPhone screen’s pixels. There is no way you will be able to see individual pixels. Every Vision Pro display gets calibrated from the factory to offer maximum colour accuracy. Display is one of the factors that pumped up the price tag.
Apple employees cheer as a customer carries a box with Apple's Vision Pro headset at the Apple Fifth Avenue store in Manhattan in New York City
The technology behind the screen is also unique. Since there are so many pixels, the computer running the show might not be able to render all of them together in high resolution all the time. There is fast eye-tracking technology at play, which is combined with foveated rendering, which is similar to the way we see. What we are looking at gets rendered in high-res while the rest (compare it with peripheral vision) is somewhat soft.
The eyes have it
Before going into implications and what it means for our future, a few words on passthrough. This is where the idea of living in reality comes in. Almost every reviewer in the US has spoken about the high level of passthrough the user enjoys and that shows Apple is thinking about the future.
Apple keeps talking a lot about AR and how they want things to be clear, overlaying things onto your real world. In the headset, they are using the highest quality camera feeds and the highest quality displays on the inside. You can feel that you’re looking through “a camera” at the real world.
As soon as you put the headset on, you notice the passthrough technology at play. In the middle of work on your Vision Pro, you want to enjoy a cup of coffee. You don’t need to take the headset off. Instead, you just look at the cup of coffee on the table, pick it up and let the beans do the talking. You can walk around and not trip over things. Or climb stairs or catch a ball that’s coming towards you. The latency is extremely low — 12ms, which involves outside light hitting the sensor to the inside image being updated and you see it. How fast is that? Eight times faster than the blink of an eye.
Making this fast delivery possible is the new R1 chip, which processes input from 12 cameras, five sensors, and six microphones to ensure that content feels like it is appearing right in front of the user’s eyes, in real time. Of course, you are looking at the real world through a camera but this is as fast as it can get at the moment.
Jest-tures?!
Filmmaker James Cameron
To select anything inside Apple Vision Pro, be it starting up an app or changing the volume while watching a film, you just need to look at it and then use hand gestures to select it. For example, you want to work on Safari, look at it and then “pinch” it. To scroll, pinch and drag. This is spatial computing at its best.
You can drag your desktop(s) into the environment — TV screen, email, music… you can get real work done. There is a virtual keyboard and the option of adding a physical keyboard. Get your MacBook out and at once, a giant desktop is before you, with all the Mac apps while the Vision Pro apps are next to it.
It’s possible to do remote desktop viewing on Meta’s Quest 3 but Apple has a different take on it. You can have a new virtual display of your Mac inside Apple Vision Pro. But it’s not about mirroring what’s already being shown on your computer. This is literally a new monitor. And you can make the “virtual monitor” as big as you want it to be.
This is a headset that is best for iPhone users; those in the Apple ecosystem. You get all the continuity features Apple keeps talking about. For example, copy text from your desktop and throw it onto the Vision Pro slate. By the way, some of the gestures to run Vision Pro will become a part of technology as other companies will pick them up.
Changing future
Since Vision Pro is about blending reality with a virtual one, opportunities are endless. Digressing, Meta went all in a couple of years ago with its metaverse game; it’s something we have read and heard about for decades. But somehow it hasn’t been able to offer enough use cases.
Jon M. Chu, director of Crazy Rich Asians
The advantage Apple enjoys is the customer base it already has. There are hundreds of millions of Apple devices around us and everyone is a potential customer.
Consider the case of immersive learning. Interacting with 3D models of an intricate piece of hardware gives you the ability to learn from anywhere. You can sit in Maniktala in Calcutta and learn about how the hardware on a fighter jet works. Or think of taking piano or guitar lessons, it will become easier and more interactive.
How about cooking? It’s something we do every day and if you want to prepare coq au vin on a weekend, the headset can help augment your cooking skills. Imagine the recipe video open somewhere in your virtual space while measurements pop up in another area. You can put timers over each of the pans you are cooking in and the oven… being a spatial computer, it can remember the location of every item you look at.
We have already seen videos of people cleaning their apartments better while wearing the Vision Pro. In one of the videos we have seen, the tedious process is gamified with coins lying on the floor and as your vacuum cleaner covers that part of the floor, the coins get sucked in. But these are simple use cases.
Let’s turn to Hollywood. Director Jon M. Chu, best known for his work on Crazy Rich Asians, was trying to edit the film adaptation of the musical Wicked when Los Angeles was flooded. Stuck at home, he decided to edit the film virtually. Chu uses Vision Pro to remotely work with his editor, Myron Kerstein. They could make and review changes on a virtual screen that was the same size as a real movie theatre screen, replicating the experience of theater-goers. He went on social media to say: “I need to repeat this out loud. I was in it for HOURS editing on a virtual giant screen (the size of a real movie screen) a major motion picture from the comfort of my house. With no headache. I can’t tell you what a revelation this was. This is big-stakes cutting-edge productivity work that is available to use today! I am still shook.”
Even James Cameron has given it full marks, saying in a Vanity Fair interview: “I would say my experience was religious, I was sceptical at first. I don’t bow down before the great god of Apple, but I was really, really blown away.”
Disney has been taken aback by the opportunities Vision Pro offers. Imagine watching a drive-in movie of Star Wars: The Force Awakens or catching Tony Stark’s home movies of Avengers: Endgame on the device. To take advantage of the level of immersion the headset offers, Disney+ has “environments” on the Vision Pro. Viewers can transform their space into one of four Disney+ environments, all brought to life by the creatives at Industrial Light & Magic (ILM). You can watch films in these special zones.
Consider live sport. You not only get to watch a match on a giant virtual screen, but there is also the opportunity to throw in two-three live matches in the same zone, besides real-time stats. The NBA is already making this possible.
The music industry will also undergo a change. Vision Pro allows musicians to play around with a range of apps designed with music creation and experience in mind. DJs can mix on realistic 3D turntables. Apps like Spool will allow editors and directors step into their music videos, allowing them to fully immerse themselves in their projects. STAGE+ brings users up and close with classical musicians, placing them in the centre of orchestral performances.
Think of the fashion industry. A designer can put on a headset to call up their design before their eyes. They can see it life-size, walk around it, just as if they were walking around a mannequin. The designer can select any aspect of a garment, like the thread used in the seams, and get it replaced. Better still, get another material and colour. Designers won’t need to wait weeks for a physical sample with the changes.
Or consider news. Bloomberg has launched the Bloomberg Pro app for Vision Pro because of the way users can play around with charts and split-screen on an infinite campus.
metric for success
Instead of looking at the number of units sold (first-generation units are flying off shelves) or customer reviews, it is necessary to speak about the number of developers who adopt the new device. The iPhone succeeded because of the large number of developers who came on board.
Vision Pro has launched with more than 600 headset-specific apps, which is impressive and a distinguishing aspect from that of Meta’s Quest. Many apps that are on the iPhone now have the opportunity to go to the next level, like Microsoft 365 and TikTok. The Vision Pro offers an infinite desktop, which is important for productivity apps as well as complex games.
end of the road for the iPhone?
No. The iPhone will have its use case while the Apple Vision Pro will be more of a productivity and entertainment tool that you will use in certain situations. It will be decades before Vision Pro achieves market saturation. In 2007, critics said the iPhone was too expensive and wasn’t useful. “It is nothing more than a luxury bauble that will appeal to a few gadget freaks,” a Bloomberg columnist wrote. Blockbuster apps will arrive on the Vision Pro App Store and only then will the game begin to change.
A typical view inside the Apple Vision Pro
Healthy tech scepticism
Developers have always celebrated the arrival of new devices to get a fair share of the gold rush. It’s about which brand makes it to the finish line first, where consumers and developers wait to cheer on. The Vision Pro is the most advanced headset in the market and Apple’s arrival is a validation of a device category. Let’s not forget that every young techie wants to see a corner that hasn’t been turned yet, hoping it brings about a change. Apple Vision Pro is at that corner. Will it be able to make the turn the product category needs?