iOS 5’s Siri-like system navigation is called ‘Assistant’, uses device-info to handle actions

Mark Gurman | Jul 22 2011 - 7:42 pm PT

[youtube http://www.youtube.com/watch?v=MpjpVAB06O4]

Apple’s purchase of Siri in early 2010 and their partnership with Nuance in 2011 has many hoping that Apple has something like speech-to-text or voice-navigation up its sleeve for iOS 5. One of the remaining advantages of Android over iOS is its system-wide Voice Actions technology.

Unfortunately, WWDC and the iOS 5 announcement came and went and nothing related to voice-navigation had been announced. Even so, the Apple-Nuance partnership has been confirmed by way of Nuance voices in Apple’s OS X Lion and Nuance speech-to-text functionality that is referenced in Apple’s internal settings modules.

But that doesn’t mean Apple isn’t hard at work at this very moment trying to cram some native OS level voice recognition technology into iOS 5 before launch.

Coupled with Nuance speech-to-text, Apple appears to be planning to take the fruit of their Siri purchase and fully integrate it into this fall’s release of iOS 5. Because these new features have yet to appear in iOS 5 on the iPhone 4 or iPhone 3GS, Apple might be saving these new features as an iPhone 5 exclusive. This would be akin to Apple’s decision to make Voice Control and video recording exclusive features to the iPhone 3GS, even though they could technically function on earlier models of the iPhone. As you can see in Siri’s promotional video above, the company advertises itself as “your virtual personal assistant.”

As you can see in the screenshot above from an Apple iPhone test unit, Apple is currently developing and testing a new iOS feature called “Assistant.” This screenshot, from a reliable source, is corroborated with our own SDK findings (below). The source did warn, though, that development is not yet completed – and just went into testing – and may not even be finished by the time the next iPhone ships.

More info after the break…

“Assistant” does not work only based on a user’s voice inputs, but Apple will take information – securely – from user’s devices like location, contact’s information, and music metadata to formulate the best information to serve a user’s request. Since Apple will be taking information such as phone contacts and location, we truly think that “Assistant” will work just like the Siri iPhone application does; but fully integrated into the operating system. We can imagine a user asking their iPhone “Assistant” to setup a movie with one of their friends. The user might say “setup movie with Mark” and based on Mark’s contact info and the user’s location data, will be able to offer tickets to a local theater and send Mark the information.

Additionally, this speech data will be crowd-sourced. Apple will give users the option to securely send their data to Apple so they can improve the service over time. This would be a process akin to Apple’s iTunes and App Store Genius functions. Additionally, this “Assistant” will be able to speak back to the user. We think that the Nuance integration here will not simply be a button a user can click in a text field to speak what they would normally type, but will also be used for the “Assistant” function to smoothly interpret and speak back information.

“Assistant” will certainly be the full package in terms of voice navigation. Apple has continuously said that they will not release something until it is perfect. This is demonstrated in Apple’s phenomenal new Notification Center system in iOS 5, seamless multi-tasking in iOS 4, and the simplest implementation of cut/copy/paste on a smartphone in iOS 3. “Assistant” will even work when a user’s device is locked (will be an option) for quick access to the functionality. Again, we need to stress that Apple is consistently experimenting, and not everything they develop makes its way into user’s hands. We sure hope ‘Assistant’ will, though.