The United States Patent & Trademark Office published an Apple patent application today (via PatentlyApple) detailing new 3D GUI concepts and touch-free, motion sensing gestures that would allow you to simply wave your hand over a device equipped with proximity sensors. This follows a patent application published in July that explores similar 3D gestures and user-interfaces, and another in September detailing 3D display and imaging technology that could lead to Kinect-like gestures on future Apple products.
The image to the right (larger version is below) shows a 3D UI environment consisting of two sidewalls, a back wall, a floor, and a ceiling. As you can see, 2D objects are posted to the back and sidewalls, while 3D objects rest on the floor of the environment. The patent mentions a “snap to” feature that appears to allow objects to move from one surface to another by changing the orientation of the 3D environment. In other words, the user’s perspective of the UI, which PatentlyApple said could be imagined as the “view from an imaginary camera viewfinder,” would change when rotation of the device is detected by its gyro sensor or accelerometer:
For example, the display environment could be displayed when the viewer holds their mobile device with the display directly facing them. In this orientation, the camera view is directly facing the back wall. As the user rotates their mobile device either clockwise or counterclockwise, the camera view is moved towards either side wall… More particularly, as the user rotates their mobile device clockwise (about the X axis of rotation), the camera view moves toward sidewall 112a.
In one scenario, Apple described gestures “made a distance above a touch sensitive display” detected by proximity sensors built into the device. We can obviously imagine the Kinect-like gestures that Apple touched upon in the previous patents mentioned above. Today’s patent, as usual, could extend to iPhone, iPad, and iPod, as well as any other device with capable GPU.
PatentlyApple noted the technology could rely on “one or more graphics processing units (e.g., NVIDIA GeForce 330M) and a 3D graphics rendering engine” such as OGRE. However, Imagination Technologies recently showed off the new PowerVR Series6 G6400 and G6200 GPUs that could theoretically provide up to 20 times better performance over chips currently being shipped in iPhone and iPad.
The patent’s approach to providing a 3D-user experience differs from the glasses-free 3D displays already shipping in consumer electronics like Nintendo’s 3DS. Although rumors have discussed the possibility of 3D display technology being used in iOS devices, a rumor from BSN that Apple will be including Sharp 3D technology in an iPad 3 is highly unlikely. Current glasses-free 3D technology is largely hindered by lack of content and health concerns.
A full-size image of the 3D environment concept is below.