How Siri works in iOS 10 for third-party apps: Rich integration with six kinds of application

Siri unlocks a new dimension of functionality in iOS 10, allowing third-party apps to integrate inline into the Siri voice assistant experience. Any developer integration expands utility of Siri dramatically but it’s important to note the limitations of the system Apple announced.

For iOS 10.0, the Siri API will only work with six kinds of application: ride booking (think Uber), messaging, Photo Search (find photos and videos in a particular app), payments (‘Send $100 to John with SquareCash’), VoIP Calling (e.g: Skype) and Workouts (for starting health and fitness activities) …

The way Siri works in iOS 10 separates developers from the core problems of voice. Apple handles the voice recognition and interpretation of queries. Siri determines user intent and decides if a particular voice query should be handled by itself or shuttled off to a third party app. This provides a consistent voice dictation interface for users as well as a strong privacy policy; Apple is always the gatekeeper of what you say.

The information the third-party app receives is limited to just the data it needs to actually do what the user wants. Siri will extract the relevant components from a query and handle just those bits (as a structured object) to the application. The app then uses various SiriKit APIs to build a response which is then displayed onscreen.

However, this means that the Siri API is not a free-for-all. As noted above, Apple has taught Siri about just a few topics for third-party integration. Ride-sharing apps like Uber or Lyft, messaging apps such as Facebook Messenger or WeChat, payments apps like SquareCash, VoIP and workout/fitness apps are in all luck. Siri supports integration with these six kinds of apps. There is also a special case for car manufacturers to enable users to adjust settings in their car through the vehicle’s special CarPlay app.

The experience for users should be great as long as you are wanting to use these types of third-party apps in Siri as Apple handles the complex voice understanding. Users must form questions with nouns like ‘WeChat’ to indicate they want to interact with a third-party app.

The SiriKit response APIs are flexible enough to allow these apps to reply with custom, rich, UI. Users will be able to complete their task inline to the Siri conversation interface; you don’t have to jump into an app.

However, the way this system is structured it leaves a lot of apps shut out, in the cold. Out of the 2 million apps in the App Store, how many fall under the support of the six categories? This is the primary downside of the system Apple devised.

Some major classes of application are not covered by the third-party Siri SDK which Siri is often used for: music apps, podcast apps, sport statistics, email, reminders and to-do lists and more. It’s unclear how this will be policed too: could an email app class itself as a text message app to slip into the Siri experience? This seems like a controversial area for App Review come September.

In future versions of iOS, it is reasonable to expect Apple to expand the number of app types that can integrate with Siri. iOS 10.1 could easily add a couple new kinds of integration. However, will it expand into certain areas that it wants to dominate with first-party services?

An obvious omission from the Siri iOS 10 implementation is third-party integration for media apps like music streaming.

With iOS 10, Apple Music will be the only way to play audio via Siri. Apple probably loves this but for users of competing music services, there is no ability for developers to be able to add Siri integration. Apps like Spotify, Pandora and Google Pay Music are left in the cold. Apple could add a new music media type for Siri SDK in future but will they? Will they keep it closed to boost up its own $9.99 streaming service, Apple Music?

This is an open question: we will have to wait and see how quickly Apple ramps the Siri SDK and expands the base of possible apps. For the public, Siri SDK launches with iOS 10 in the fall.