Apple patent application details AR/VR headset with gesture tracking and more

Apple has recently filed a patent for a mixed-reality headset device that would “provide 3D virtual views of a user’s environment augmented with virtual content.” The patent was first detailed by Variety.

Apple first filed this patent application back in March, but it was only published within in the last week. The patent describes how an augmented reality headset could capture the real world using outward-facing cameras, then present that to the user on a display. The user could interact with that environment:

“In some embodiments, the world sensors may include one or more “video see through” cameras (e.g., RGB (visible light) video cameras) that capture high-quality video of the user’s environment that may be used to provide the user with a virtual view of their real environment.”

The patent also details how the head-mounted display could include sensors that read inputs from the users, such as expressions, eye movement, and head gestures.

The head-mounted display may include sensors that collect information about the user’s environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user’s expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors

Last year, CNET reported that Apple was developing an AR/VR headset capable of pushing a 16K resolution. Just a few months ago, reliable Apple analyst Ming-Chi Kuo said that Apple would start manufacturing its AR headset by Q4 2019 or Q2 2020. On the other hand, however, there have been reports out of the supply chain that Apple has canceled its plans for a mixed reality headset. Whether or not that turns out to be true remains to be seen.

Would you be interested in a mixed reality headset from Apple? How would it need to differentiate itself from existing options from companies like Oculus? Let us know your thoughts down in the comments!