Apple researching universal remote that customizes UIs intelligently

This is not the first time we have received hints that Apple is working on an innovative universal remote control for controlling TV and video content. In January, we told you that Apple was researching a touchscreen remote with adaptable user interfaces. The invention would essentially allow button layouts stored in the cloud or in a device (such as a TV) to be wirelessly and seamlessly beamed to the controller’s UI. The concept would alleviate the “table full of remotes” scenario Steve Jobs described at D8.

Today, a new patent application published by the United States Patent & Trademark Office and detailed by PatentlyApple gives us even more insight into what Apple’s universal remote concept could become. In the newly discovered patent application, Apple details a remote that is capable of displaying customized controls for various devices by simply taking a picture of the device. Apple would send the picture to iCloud, analyze it, and beam a UI or button layout to the remote that works for your TV. PatentlyApple explained:

One of Apple’s wild new universal remote features works like this: you take a photo of your current TV remote and it sends it to iCloud for analysis. It then sends a “virtual copy” of your remote, functionality and all, to your iPhone. You’re now able to control your current TV with Apple’s advanced universal remote and enjoy all of Apple’s TV remote features and likely tie-in products and services.

Apple’s remote–which the patent mentions could be integrated into an iOS device with IR capabilities—could include one or more cameras, microphones, and sensors capable of detecting changes in volume, indicator lights on devices, etc. These could decide the current state of a device, allowing the remote’s functionality to adapt, for example, to what is onscreen at any given moment.

Rather than having to snap a photo of your TV, as described above, the camera could also detect logos of a device (such as a Sony TV logo) to pull and create a virtual copy of the device’s physical remote. As the report explained, “the iPhone may be configured to create virtual buttons that correspond in both appearance and functionality as the native TV remote.”

The report also described the remote’s cameras being capable of detecting shapes or colors (such as an HBO logo):

the image may be processed to determine shapes and/or characters on a television screen. The shapes and characters may be used to determine that the television is displaying content from a particular source. If you think of it, that translates in-part to reading logos on screen from sources such as ABC, NBC or HBO for instance… a television having a particular aspect ratio may be recognized based on it proportions and/or its size relative to other objects. A brand name or logo may be recognized for a particular DVD player, or DVR (e.g., TiVo). Moreover, a shape and/or particular arrangement of a face of a device may be recognized (e.g., a Nintendo Wii video game console).

Of course, with Apple continuing its hobby set-top box project with a recently refreshed Apple TV, and rumors of an Apple-branded HDTV, it is possible Jobs’ vision of finally cracking television and users no longer having to “fiddle with complex remotes” also included a revolutionary remote control.

Related articles

• Analyst: Apple could use ‘iTV’ moniker for HDTV, partner with carriers for programming (9to5mac.com)
• Universal and Fox movies blocked from iCloud on new Apple TV? (9to5mac.com)
• WSJ: Apple discussing voice-powered TV set, enhanced AirPlay, iCloud DVR service (9to5mac.com)