Skip to main content

Apple’s mixed reality headset project challenges explained

Rendering by Antonia De Rosa. Edited by 9to5Mac.

Since beginning its efforts in 2015, Apple’s work on a mixed reality headset has been hit with a range of challenges that have pushed back the potential launch date by years. Now a new report from The Information shares what the major roadblocks have been from inside sources.

The Information was able to talk with 10 people close to Apple’s mixed reality headset project, most of whom worked on the team. Going all the way back to 2016, just one year after development started, the sources say Apple’s board was presented with a demo of very early prototypes to try and “build support at Apple’s highest levels for bigger investments.”

Former Vice President Al Gore, then–Disney CEO Bob Iger and other Apple board members walked from room to room, trying out prototype augmented and virtual reality devices and software. One of the gadgets made a tiny digital rhinoceros appear on a table in the room. The creature then grew into a life-size version of itself, according to two people familiar with the meeting. In the same demo, the drab surroundings of the room transformed into a lush forest, showing how users could seamlessly transition from AR, in which they can still view the physical world around them, to the more immersive experience of VR—a combination known as mixed reality.

Sources say some of the prototypes were “cobbled together” and ran Microsoft Windows while others were modified HTC Vives or other existing headsets. However, one of them was so heavy it required a “small crane” to hold it “so the Apple board members could wear it without straining their necks”

While that doesn’t sound like an overly impressive experience, Apple’s board was convinced to support more development funding (another factor here was Apple’s concern about Facebook’s progress in the space).

Going forward, the biggest challenges have been technical and one factor that’s allegedly been at play is a CEO Tim Cook supporting the work, but not being a “champion” and highly involved in it.

Technical challenges have been the biggest factor in the delays, as has been the case in the past for Apple’s most ambitious new products, such as the iPhone. But the Apple smartphone also had a singularly influential figure in Apple co-founder Steve Jobs to midwife it.

While Apple’s current CEO, Tim Cook, supports the headset project, he hasn’t been as active in the effort as Jobs was with the iPhone’s development, according to five people familiar with the project. For example, he rarely visits the group at its offices away from the main Apple campus, those people said. The lack of a honcho of Cook’s stature to champion the headset, code-named N301, has made it harder at times for it to compete with other products such as the Mac and iPhone for head count and engineering resources, the people said.

Another challenge came from Jony Ive. When the headset team founder and leader Mike Rockwell was working on getting buy-in from Apple’s various teams to help with the development, his team was shut down on the idea of making it a VR headset. Ive’s team also pushed back on “practical uses” and doubted consumers would want to wear headsets for any considerable amount of time.

Rockwell, Meier and Rothkopf soon encountered pushback from Ive’s team. The three men had initially wanted to build a VR headset, but Ive’s group had concerns about the technology, said three people who worked on the project. They believed VR alienated users from other people by cutting them off from the outside world, made users look unfashionable and lacked practical uses. Apple’s industrial designers were unconvinced that consumers would be willing to wear headsets for long periods of time, two of the people said.

That ended up birthing the idea of a mixed reality headset:

The men came up with a solution to address the concerns of Ive’s team. For example, they proposed adding cameras to the front of the headset so that people wearing the device could see their surroundings, said the three people. But the feature that ultimately sold the industrial designers on the project was a concept for an outward-facing screen on the headset. The screen could display video images of the eyes and facial expressions of the person wearing the headset to other people in the room.

These features addressed the industrial design group’s worries about VR-induced alienation—they allowed other people in a room to interact and collaborate with a person wearing a headset in a way not possible with other VR gear. For years, the existence of such a display, internally code-named T429, was known only to a small circle of people even within Rockwell’s group.

The Information’s report hints that a follow-up piece will cover a “pivotal moment for the Apple headset” that occurred in 2019. That lines up with when Jony Ive “balked” at the idea of selling a headset that required a base station and the team pivoted to working on a less powerful, but more independent AR/VR device.

The latest expectation is that Apple could announce its mixed reality headset in 2023. As far as price, we’ve heard reports that it could sell from above $2,000 to $3,000.

FTC: We use income earning auto affiliate links. More.

Roborock Q5+ robot vacuum
You’re reading 9to5Mac — experts who break news about Apple and its surrounding ecosystem, day after day. Be sure to check out our homepage for all the latest news, and follow 9to5Mac on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel

Comments

Author

Avatar for Michael Potuck Michael Potuck

Michael is an editor for 9to5Mac. Since joining in 2016 he has written more than 3,000 articles including breaking news, reviews, and detailed comparisons and tutorials.